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PREFACE 


This  report  on  the  Military  Operations  Research  Society's  Mini-Symposium  on  Distributed 
Interactive  Simulation  (DIS)  provides  an  excellent  view  of  the  impact  of  DIS  on  Military 
Operations  Research  (MOR)  in  September  1992.  It  is  a  valuable  aid  to  determining  the  influence 
of  MOR  on  the  development  of  DIS  in  the  three  years  since  the  Symposium  was  held. 

Since  that  time  Distributed  Interactive  Simulation  technology  has  matured  rapidly.  Many 
of  the  conclusions  of  this  report  on  Military  Operations  Research  and  DIS  remain  valid.  Many 
of  the  report's  statements  on  the  technology  of  DIS  and  the  uses  of  DIS  need  to  be  revised  in  light 
of  current  technology.  This  has  not  been  done  within  the  report's  context.  The  primary  reason 
is  to  maintain  accuracy  about  the  state  of  knowledge  of  DIS  which  was  reported  at  the 
Symposium. 

The  document  is  abbreviated  to  an  Executive  Summary,  Introduction,  and  the  Final  Reports 
of  Working  Groups  1  - 10.  It  contains  neither  papers  presented  at  the  Symposium  nor  Working 
Group's  preliminary  slides  presented  to  the  Symposium.  A  reason  for  brevity  is  to  bring  the 
publication  process  to  a  conclusion. 

An  excellent  recent  reference  to  Advanced  Distributed  Simulation  (ADS)  and  Distributed 
Interactive  Simulation  is  the  June  1995  special  issue  of  PHALANX,  The  Bulletin  of  Military 
Operations  Research,  on  ADS/DIS.  Discussed  there  are  ADS/DIS  issues  of  architecture, 
conceptual  modeling,  current  technology,  experimental  approaches,  future  technology,  and 
verification,  validation  and  accreditation.  Statements  about  the  technology  of  DIS  in  this  report 
should  be  compared  to  those  in  PHALANX. 


Julian  Palmore 
Editor 
July  1995 
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EXECUTIVE  SUMMARY 


The  Mini-Symposium  allowed  participants  in 
Military  Operations  Research  (MOR)  and  in 
Distributed  Interactive  Simulation  (DIS) 
technology  to  meet  and  exchange  views  on 
ways  in  which  this  developing  DIS  technology 
can  influence  future  MOR. 

Mini-Symposium  participants  were  enthusias¬ 
tic  about  the  potential  of  DIS  while  remaining 
both  cautious  about  its  limitations  and  desirous 
of  further  research. 

Comments  on  DIS  limitations  or  needs  for 
further  research  in  this  document  do  not  imply 
that  work  in  an  area  has  not  already  begun. 
The  implication  is  that  appropriate  efforts  must 
continue  with  concerned  communities  actively 
sharing  responsibility.  A  goal  of  DIS  is  to 
integrate  seamlessly  on  a  network  virtual 
simulations  (human-in-the-loop,  virtual  real¬ 
ity),  constructive  simulations  (engineering, 
force-on-force,  or  other  closed-form),  and  live 
simulations  (field  tests  or  exercises)  and  their 
interactions. 

APPLICATIONS  AND  LIMITATIONS 

Development.  Many  DIS  applications  were 
identified  as  appropriate  for  system  develop¬ 
ment.  Perhaps  the  most  important  were  deter¬ 
mining  how  to  use  a  new  system  effectively  on 
the  battlefield  and  how  to  integrate  it  into  the 
existing  force.  Other  early  development  appli¬ 
cations  included  requirements  generation; 
trade-off  analysis;  organizational,  operational 
concept,  and  other  force  structuring  experi¬ 
mentation;  development  of  tactics,  techniques, 
and  procedures;  work  on  systems  integrations; 
assessment  in  the  human  factors  area;  and 


planning  for  later  operational  tests. 

When  a  system  offers  a  radically  new  capabil¬ 
ity  for  DIS  there  are  many  useful  research 
applications.  Examples  of  research  topics  are 
command  and  control,  combat  support  and 
service  support,  behavioral  phenomena,  alter¬ 
nate  technologies,  development  of  consistent 
measures  of  effectiveness  or  performance  for 
all  communities,  and  calibration  of  different 
aggregate  level  simulations. 

DIS  applications  are  limited  when  the  physics 
of  the  system  under  test  are  not  well  under¬ 
stood.  DIS  has  no  direct  applications  for 
engineering  testing  of  the  basic  physics  under¬ 
lying  system  performance.  The  necessary  high 
resolution  engineering  models  need  not  neces¬ 
sarily  be  distributed  or  interactive.  It  was 
thought  that  DIS  might  give  us  insights  on 
how  to  exploit  performance  changes  in  battle. 

Planning  or  rehearsal.  Battle  planning  or 
rehearsal  may  be  applications  of  DIS.  Terrain 
"reconnaissance"  and  analysis  were  considered 
promising  provided  that  graphics  capabilities 
improve.  Semi-automated  forces  (SAFOR) 
may  be  worthwhile  for  course-of-action  analy¬ 
sis  or  rehearsal  options. 

These  DIS  applications  will  be  limited  until 
further  improvements  are  implemented.  Over 
the  next  five  years,  the  necessary  improvement 
in  simulation  network  (SIMNET)  graphics 
technology  is  expected  to  become  a  reality. 
Then,  the  more  important  issue  will  be  how 
much  the  tactical  implication  of  systems  might 
be  altered  by  operators  based  upon  erroneous 
data  in  the  system  representation.  Running 
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simulators  with  unclassified  data  is  a  particular 
concern.  An  interesting  warning  was  voiced 
against  the  danger  of  overly  specific  scenarios 
that  might  influence  units  to  be  less  flexible  for 
unanticipated  situations. 

Test  and  evaluation  (T&E).  DIS  could 
provide  alternative  means  of  evaluation  to 
testing  when  particular  conditions  (e.g.,  in  the 
case  of  nuclear,  biological,  chemical,  and  space 
systems)  are  limiting  or  testing  is  otherwise 
constrained  (e.g.,  environmental,  safety,  politi¬ 
cal,  electromagnetic  emanations,  cost).  When 
DIS  is  accredited  as  a  test  tool,  its  proper 
focus  would  be  on  comparative  human  factors 
analysis.  A  consistent  DIS  application  might 
produce  developmental  or  user  test  results  to 
support  the  credibility  of  Cost  and  Operational 
Effectiveness  Analysis  (COEA)  and  for  front 
end  screening  of  a  next-generation  system  or 
set  of  mission  requirements.  DIS  may  also 
support  multi-service  and  joint  evaluations 
(usually  for  high-priced  systems)  requiring 
combined  arms  procedures  before  the  actual 
systems  are  produced. 

DIS  T&E  applications  are  limited  when  the 
system  under  test  does  not  need  to  interact 
with  other  systems,  the  system  is  easy  and 
inexpensive  to  test  normally,  or  the  system  is 
one  of  the  Major  Automated  Information 
Systems  (MAIS)  or  any  other  information 
management  systems. 

Training.  Training  has  the  widest  set  of 
recognized  DIS  applications.  DIS  shows 
considerable  promise  for  carefully  controlled 
training  appli cations,  from  small  units  to  joint 
task  force  operations  by  allowing  reactive 
activities  with  visualization  to  enhance  credi¬ 
bility  and  playback  for  after-exercise  reviews. 
Additional  benefits  will  likely  accrue  from  the 
linking  of  diverse  simulators  and  simulations  as 
less  costly  alternatives  to  live  exercises.  Con¬ 


structive  simulations  are  already  important 
tools  for  higher  echelon  training  exercises. 

There  will  remain  training  events  that  are  best 
served  by  live  instead  of  virtual  or  constructive 
simulations.  For  example,  individual  weapon/ 
vehicle  skills  or  crew  task  proficiency  training 
are  best  accomplished  by  field  exercise  or  part 
task  trainers,  rather  than  DIS.  Guidelines  for 
using  simulation  for  different  training  events 
should  be  developed  and  provided  to  the 
training  community.  One  must  consider  that 
there  is  a  significant  risk  of  negative  training 
when  simulators  are  used  for  tasks  in  which 
their  fidelity  is  inadequate. 

It  should  be  noted  that  there  is  no  training 
objective  for  a  DIS  application  covering  all 
echelons  from  "god  to  squad"  due  to  differ¬ 
ences  in  planning  horizons,  tempo  of  opera¬ 
tions,  information  needs,  and  training  focus. 
In  fact,  it  is  best  that  various  echelons  pause, 
reexamine,  or  accelerate  their  exercises  as 
necessary  for  specific  training  objectives, 
rather  than  be  lock-stepped  inappropriately  by 
DIS  with  other  echelons. 

NEEDED  ENHANCEMENTS,  RE¬ 
SEARCH  AREAS,  RECOMMENDA¬ 
TIONS 

Capacity.  Although  the  capacity  of  a  DIS 
network  depends  upon  other  requirements, 
more  research  is  needed  on  how  to  size  the 
network  requirements.  The  potential  of  tech¬ 
niques  such  as  data  compression,  combining 
like  protocol  data  units  (PDU),  multicast, 
smart  gateways,  fibre  chord,  token  rings,  and 
more  advanced  update  algorithms  must  be 
assessed.  Latency  (the  delay  experienced 
between  an  entity  state  change  and  the  time 
other  entities  on  the  network  become  aware  of 
that  change)  must  be  considered  in  terms  of 
transmission  time,  PDU  processing  time, 
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encryption  constraints,  and  other  factors. 
How  more  capable  pre-  and  post-processors 
can  be  added  should  be  explored. 

DIS  capacity  reflects  upon  DIS  availability. 
Participants  felt  DIS  should  be  available  to 
anyone  with  a  valid  need  and  a  means  to  pay. 
The  general  consensus  was  DIS  will  be  en¬ 
riched  by  a  diversity  of  users.  Charges  should 
offset  the  cost  of  DIS  operations  and  mainte¬ 
nance,  and  discourage  frivolous  demands  for 
access.  Another  concern  impacting  availability 
is  DIS  reliability  which  must  be  increased 
before  critical  exercises  and  expensive  testing 
of  procedures  can  be  safely  attempted.  All 
components  of  DIS,  including  individual 
simulators,  communications,  computers,  and 
software  must  be  made  more  reliable  for  DIS 
to  be  used  at  its  full  functional  capacity. 

Cost-benefit  analysis.  How  to  measure  costs 
and  benefits  must  be  practically  addressed  to 
make  correct  decisions  regarding  DIS.  For 
example,  cost-benefit  comparisons  should  be 
done  between  DIS  supported  training  events 
and  other  training  options.  Generally,  the 
range  of  potential  applications  and,  thus, 
benefits  is  much  larger  than  the  very  limited 
applications  which  currently  exist.  Moreover, 
most  evidence  of  benefits  is  anecdotal.  Broad¬ 
ly  speaking,  there  are  no  major  impediments  to 
estimating  costs  of  desired  DIS  applications, 
but  the  effort  has  not  yet  been  made.  Defining 
the  problem  is  even  more  important  in  a  dis¬ 
cussion  of  DIS  where  there  is  clearly  the 
danger  of  becoming  infatuated  with  the  tech¬ 
nology  and  end  up  applying  it  inappropriately. 

Defining  DIS.  The  traditional  view  of  DIS  is 
characterized  as  human-in-the-loop  simula¬ 
tions  interacting  with  each  other  to  perform 
common  tasks.  A  broader  definition  is  needed. 
The  new  definition  of  DIS  can  and  should  refer 
to  "any  combination  of  virtual,  constructive 


and  live  simulations  that  are  distributed  over  a 
network  and  interact  through  standardized 
protocols."  While  this  definition  encompasses 
SIMNET  as  originally  implemented,  it  would 
also  include  situations  such  as  Corp  Battle 
Simulation  linked  to  T  AC  SIM  (an  intelligence 
sensor  simulation),  SIMNET  linked  to  EA¬ 
GLE  (a  corps  level  command  and  control 
combat  simulation),  SIMNET  linked  to  units 
exercising  at  the  National  Training  Center,  and 
similar  examples  from  the  Navy  and  the  Air 
Force. 

Environment.  DIS  needs  improvements  in 
the  fidelity  and  resolution  of  environmental 
representations  for  nature  (weather,  illumina¬ 
tion,  terrain,  clutter),  man-made  factors  (ob¬ 
scurants,  dynamic  terrain,  electromagnetic, 
artillery  and  effects),  and  the  technical  environ¬ 
ment  of  the  simulation  (higher  fidelity  displays, 
algorithms,  protocols,  signatures,  semi-auto¬ 
mated  forces/computer  generated  forces). 
"Good"  fidelity  and  resolution  are  determined 
by  particular  users  who  have  differing  require¬ 
ments,  so  either  the  strictest  requirements  must 
be  satisfied,  or  simulations  should  be  designed 
so  that  their  environmental  realism  can  vary. 

Logistics  simulation.  Logistics  elements 
should  be  included  in  DIS  training  at  brigade 
and  battalion  level  with  visible  and  active 
support  elements.  SAFOR  should  preclude 
excessive  manpower  or  simulator  require¬ 
ments.  Also,  to  alleviate  the  disparity  in  simu¬ 
lation  time  between  logistics  problems  and 
tactical  exercises,  logistics  simulations  must  be 
able  to  run  in  a  faster-than-real-time  mode. 
For  example,  a  battalion  of  tank  crews  would 
gain  little  value  if  they  were  to  end  up  in  divi¬ 
sion  reserve  for  a  week  long  exercise,  doing 
nothing  but  moving  from  assembly  area  to 
assembly  area.  Additionally,  DIS  should  be 
considered  for  use  in  operational  training  at 
logistics  elements  such  as  training  equipment 
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operators  at  a  port  facility.  Further  research  is 
needed  to  determine  the  levels  of  fidelity  and 
resolution  appropriate  for  logistics  portions  of 
battlefield  simulations,  and  the  logistics  com¬ 
munity  must  begin  to  develop  an  appropriate 
"end-to-end"  module  for  the  current  set  of 
virtual  simulations.  This  module  should  span 
service  responsibilities  including  both  airlift 
and  sealift.  Except  for  analyses  for  demands 
on  recovery  and  combat  damage  repair  assets, 
little  of  the  actions  at  brigade  or  battalion  as 
represented  in  SIMNET-like  simulations 
directly  affects  the  logistics  training  audience, 
which  is  better  trained  by  constructive  models 
in  DIS  having  more  aggregation.  Actually, 
when  significant  numbers  of  SAFOR  are  in 
use,  the  DIS  is  operating  more  at  an  aggregate 
level.  Furthermore,  logistics  may  require  many 
iterations  to  develop  levels  of  confidence, 
which  would  be  easier  to  accomplish  with 
constructive  simulation. 

Management.  A  significant  management 
concern  is  how  to  budget  and  schedule  as 
more  players  become  involved  with  DIS. 
Master  planning  and  scheduling  of  utilization 
of  DIS-related  resources  (network,  nodal 
simulations,  players)  over  a  planning  horizon 
of  several  years  will  be  critical.  One  working 
group  proposed  a  dedicated  scheduling  mecha¬ 
nism  similar  to  the  CINCs'  exercise  schedule 
or  the  Army's  Five-Year  Test  Program 
(FYTP).  DIS  needs  a  network  manager  for 
network  scheduling,  assuring  asset  readiness, 
coordination  of  user  needs,  planning  enhance¬ 
ments,  integration,  and  budgeting.  In  the  near 
term,  this  network  manager  should  also  adver¬ 
tise  DIS  capabilities  to  potential  users.  In  the 
future,  the  network  manager  would  serve  as 
the  single  point  of  contact.  Each  installation 
with  a  node  should  also  have  a  site  manager. 
This  person  would  schedule  use  of  the  node 
and  possibly  use  of  local  DIS  assets  for  user 
applications.  There  must  be  a  disciplined 


process  for  accessing  DIS  resources,  de-con¬ 
flicting  competing  demands  for  their  use,  and 
reimbursing  scheduled  users  when  their  plans 
are  disrupted  by  out-of-cycle  users  (queue 
jumpers).  Other  management  concerns  include 
the  distribution  and  maintenance  of  common 
data  bases  and  module  (components)  under 
sound  configuration  management,  more  com¬ 
prehensive  documentation  for  components 
which  cannot  be  examined  first  hand  by  a  user, 
establishing  libraries  for  remote  access  of  such 
documentation,  and  monitoring  capabilities 
such  that  when  a  critical  node  drops  off  of  the 
net  due  to  a  system  failure,  the  DIS  network 
controller  can  take  an  appropriate  corrective 
action  immediately. 

Semi-Automated  Forces  (SAFOR)/  Com¬ 
puter  Generated  Forces  (CGF).  SAFOR 
validity  was  discussed  in  great  detail.  SAFOR 
should  be  indistinguishable  from  manned 
systems  in  DIS  to  best  support  training.  A 
data  collection  capability  is  needed  to  capture 
simulated  communications  among  SAFOR  as 
well  as  between  SAFOR  and  live  units.  The 
decision  processes  affecting  SAFOR  actions 
must  be  captured  and  be  available  for  debrief¬ 
ing.  Human  factors  such  as  morale,  fatigue, 
and  indecision  must  be  exhibited  by  SAFOR. 
Tactics,  perceived  objectives,  rules  of  engage¬ 
ment,  and  decision  processes  must  be  adapt¬ 
able  to  the  specific  enemy  consistent  with  the 
training  scenario.  The  span  of  control  of 
SAFOR  operators  must  be  increased  signifi¬ 
cantly  so  that  units  can  train  in  large  scale 
operations  without  excessive  controller  costs. 
Identification  issues  of  friend,  foe  or  neutral 
systems  must  not  be  automatically  resolved 
either  among  or  between  SAFOR.  Terrain 
avoidance  and  dynamic  terrain  representation 
are  improvements  necessary  to  realize  the  full 
training  potential  of  DIS.  The  need  to  validate 
the  performance  of  the  SAFOR  should  be  an 
integral  part  of  the  basic  design  to  ensure  that 
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the  necessary  data  is  explicitly  represented  and 
captured  for  review.  If  SAFOR  is  to  become 
a  high  resolution  model  of  analytic  choice, 
there  is  a  need  for  a  single  analyst  to  run  or 
interrupt  the  model  repeatedly  without  a  staff 
guiding  the  individual  vehicles  as  they  move 
through  the  battle.  This  is  not  meant  to  imply 
that  the  SAFOR  would  be  used  purely  in  an 
"unattended"  mode.  Most  certainly,  a  reason¬ 
able  analyst  would  carefully  review  the  new 
battles  for  validity  against  a  sound  tactical  plan 
at  the  division/brigade/battalion  level.  SAFOR 
should  be  enhanced  to  contain  "pre-blessed" 
scenarios,  rules  of  engagement,  tactics,  doc¬ 
trine,  etc. 

Security.  Multi-level  security  issues  must  be 
resolved  for  various  classified  DIS  applica¬ 
tions.  Smart  gateways  and  security  encryption 
may  overcome  problems,  but  users  should  be 
aware.  Viruses  transmitted  via  PDUs  is  a  new 
danger  for  which  scanners  and  protective 
techniques  must  be  used.  Multi-level  security 
capabilities  must  be  developed  to  more  pre¬ 
cisely  manage  access  and  use  of  classified 
material  on  the  network  would  be  a  consider¬ 
able  assist  in  dealing  with  the  use  of  unclassi¬ 
fied  ("inaccurate")  data.  Prudent  management 
of  network  access,  and  encryption  may  be 
sufficient  to  protect  a  firm's  proprietary  infor¬ 
mation  placed  at  risk  while  using  DIS. 

Standards.  The  need  for  additional  work  on 
standards  is  widely  recognized.  One  new  area 
for  standardization  is  linking  actual  systems 
through  data  buses  to  the  DIS  network.  Such 
linking  would  facilitate  the  verification,  valida¬ 
tion,  and  accreditation  process.  This  is  a  task 
which  requires  much  research,  but  one  that 
may  pay  great  dividends  in  terms  of  credibility. 
A  standard  that  needs  implementation  is  for  a 
common  time  reference  such  as  the  Global 
Positioning  System.  Work  at  Armstrong  Labs 
identified  the  need  for  additional  data  elements 


to  be  incorporated  in  each  packet  describing  a 
fast  moving  airborne  vehicle  due  to  increased 
speed  and  three  dimensional  travel.  Consider¬ 
ations  for  DIS  training  and  how  SAFOR 
replicates  performance  make  this  a  prime 
research  area  under  DIS  standards  to  address 
joint  operations.  Also,  it  will  sometimes  be 
necessary  to  transmit  and  gather  data  not 
envisioned  in  current  PDUs.  Research  is 
needed  for  flexible,  variable  resolution  PDU 
standards  to  reserve  such  capabilities  even  for 
non-existent  functionalities  such  as  a  death  ray 
or  transporter  beam. 

CONCERNS 

Resourcing.  Many  participants  felt  that  OSD 
should  fund  the  continued  evolution  and  oper¬ 
ations  and  maintenance  (O&M)  of  DIS  which 
are  "in  common"  to  OSD,  JCS,  CINCs,  and 
the  Services.  Customers  with  unique  needs  or 
applications  should  pay  their  own  way.  The 
funding  issue  was  particularly  worrisome 
because  needs  exceed  the  expertise  of  most 
organizations  and  will  require  contracted 
augmentation.  Also,  a  key  potential  of  DIS  is 
in  early  development,  before  a  funded  program 
exists.  "Where  will  the  funds  come  from?" 
was  the  primary  question.  It  had  no  satisfac¬ 
tory  answer.  The  use  of  DIS  can  place  signifi¬ 
cant  demands  for  personnel  resources;  i.e., 
participants,  as  well  as  support  staff  such  as 
controllers  and  data  base  experts.  Full-time 
support  staff  should  be  identified  for  all  DIS 
centers.  The  extent  of  the  training  program 
required  for  personnel  who  will  operate  DIS 
has  not  been  adequately  addressed  yet.  There 
are  already  indicators  that  the  Army's  growing 
capability  in  virtual  simulations  is  not  keeping 
pace  with  the  demand  for  access  to  it.  Fur¬ 
thermore,  if  DIS  is  to  become  a  viable  test 
adjunct,  a  new  OSD  directive  is  needed  which 
will  require  all  battlefield  system  requests  for 
proposals  (RFPs)  to  state  that  simulators  must 
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be  DIS-compatible.  The  responsibilities  must 
be  clearly  delineated  in  the  Test  and  Evalua¬ 
tion  Master  Plan  (TEMP).  The  potentials  of 
DIS  are  dependent  upon  the  degree  that  the 
acquisition  strategy  invokes  DIS  consistently 
throughout  all  milestones. 

Requirements.  So  far,  DIS  has  resulted  from 
"technology  push."  For  its  full  potential  to  be 
reached,  it  needs  more  "requirements  pull." 
Work  on  identifying  requirements  is  underway 
and  should  reduce  the  danger  from  warfighters 
being  active  DIS  participants  and  becoming 
infatuated  with  the  DIS  capabilities  before  they 
receive  an  appropriate  level  of  more  rigorous 
scrutiny.  It  also  reduces  the  potential  danger 
of  a  mismatch  between  expectations  and  the 
near-  or  long-term  results  of  DIS  programs.  It 
is  equally  important  that  existing  tools  and 
methods  not  be  prematurely  or  inappropriately 
rejected  because  of  misperceptions  regarding 
the  benefits  and  costs  of  DIS  applications. 
"Turf1  obstacles  must  still  be  overcome  to 
clarify  DIS  requirements.  One  example  is  how 
to  direct  joint  DIS  applications  such  as  combat 
developments.  A  less  obvious  example  stems 
from  the  concurrent  development  of  DIS  and 
the  system  being  simulated.  It  is  likely  DIS 
will  continuously  represent  the  latest  version 
of  a  system,  but  who  will  have  the  responsibil¬ 
ity  for  configuration  management,  fielding  new 
versions,  and  deciding  what  system  perfor¬ 
mance  will  be  represented  in  a  particular 
version  for  a  specific  application?  It  is  not 
presently  clear  how  to  satisfy  requirements. 
Should  future  versions  of  DIS  be  backwards 
compatible  to  allow  users  to  make  use  of  new 
capabilities  in  future  versions  with  minimum 
adjustment? 

Risks.  Expectations  that  DIS  is  immediately 
available  are  based  upon  demonstrations  and 
prototypes  in  limited  numbers  with  limited 
capabilities.  If  past  experience  holds,  hard¬ 


ware  and  software  which  are  affordable  and 
fieldable  in  larger  quantities  may  not  be  avail¬ 
able  for  a  while.  Also,  the  full  costs  of  DIS 
have  not  yet  been  determined  and  many  costs, 
particularly  operating  costs  such  as  configura¬ 
tion  management,  data  management,  support 
staffing,  verification,  and  validation,  have  not 
yet  been  addressed  in  detail.  These  "hidden" 
costs  may  be  large  and  may  not  naturally  fall 
into  any  one  organization's  budget.  Addition¬ 
ally,  the  development  of  models  and  simula¬ 
tions  for  DIS  applications  will  pose  a  series  of 
challenges  since  it  will  be  necessary  to  break 
new  ground.  Validation  of  these  new  develop¬ 
ments  will  be  very  challenging.  SAFOR  is  a 
good  example  of  this.  The  costs  of  specific 
SAFOR  applications  and  their  associated 
benefits  depend  upon  the  validity  and  scope  of 
SAFOR  capabilities,  but  SAFOR  technology  is 
not  yet  mature.  Similarly,  the  question  of 
matching  fidelity  and  resolution  to  applications 
remains  open  and  must  be  resolved.  Finally, 
among  developers  of  DIS  and  among  potential 
users,  there  are  perceptions  that  DIS  replaces 
tools  currently  used  in  all  applications.  This 
perception  is  not  correct.  DIS  will  augment 
other  tools  and  approaches.  Care  must  be 
taken  if  we  are  to  avoid  premature  acceptance 
or  rejection  of  DIS  in  various  applications. 

VERIFICATION,  VALIDATION  AND 
ACCREDITATION 

Much  of  the  discussion  during  the  mini-sym¬ 
posium  dealt  with  the  thorny  issue  of  verifica¬ 
tion,  validation,  and  accreditation  (W&A)  of 
DIS  applications.  W&A  appears  to  be  the 
single  issue  that  appropriately  spans  nearly  all 
communities'  interests.  The  challenges  of 
W&A  embody  the  challenges  of  DIS  overall. 
To  understand  this,  consider  what  must  be 
learned  to  credibly  apply  DIS  as  envisioned 
and  examine  how  W&A  will  be  associated 
with  future  DIS  management  structure. 
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As  DIS  technology  expands,  research  will  be 
needed  to  develop  new  procedures  and  theory 
to  meet  the  demand  for  W&A  of  DIS.  In 
particular,  aspects  of  DIS  requiring  research 
include  methods  for  verifying  "boundary 
crossing"  interfaces  between  characteristically 
different  DIS  components,  requirements  for 
certifying  personnel  playing  a  human-in-the- 
loop  role,  procedures  for  calibration  of  the 
system  components  with  different  levels  of 
aggregation,  experimental  design  methods  to 
support  interoperation  of  statistically  dissimilar 
simulations,  procedures  to  evaluate  the  physi¬ 
cal  (electronic)  limitation  or  restriction  of  a 
particular  network  architecture.  It  will  be 
necessary  to  identify  and  design  experiments 
and  test  cases  for  DIS  applications  which  can 
furnish  information  to  support  W&A.  Such 
research  efforts  should  be  documented  through 
journal  articles,  issue  papers  or  position  papers 
with  the  ultimate  goal  of  the  development  of 
primers  or  handbooks  to  instruct  the  analyst 
DIS  user  how  to  accredit  a  system  for  particu¬ 
lar  analytic  application.  This  research  will  be 
necessary  for  the  wider  military  community  to 
accept  DIS  as  credible. 

While  research  continues,  there  will  also  be 
management  challenges.  For  single  service 
applications  in  the  future,  pre-accredited 


modules  ("off  the  shelf')  should  be  developed 
or  archived  for  classes  of  DIS  applications. 
Pre-accreditation  implies  an  agency  serving  as 
the  repository  of  W&A  and  other  documen¬ 
tation  for  "approved"  modules.  For  multi¬ 
service  DIS  applications,  the  "user"  (lead 
agency)  should  review  W&A  of  all  net  com¬ 
ponents,  prepare  the  W&A  documentation, 
describe  the  limitations  of  the  DIS  application, 
and  prepare  inter-service  MOAs  as  needed. 
The  management  to  accomplish  these  W&A 
efforts  must  be  established  in  parallel  with 
continued  DIS  activities  on  many  fronts. 

Concentrating  on  how  W&A  will  be  con¬ 
ducted  for  the  user  communities  is  a  good 
method  to  plan  for  how  and  to  what  extent 
DIS  can  be  exploited.  It  will  be  key  to  ensur¬ 
ing  that  DIS  will  be  used  appropriately  for  its 
various  applications.  Although  the  DIS  tech¬ 
nology  is  believed  by  many  to  have  the  poten¬ 
tial  to  be  a  key  operations  research  tool  of  the 
future,  it  is  clear  that  many  improvements  are 
needed  to  meet  the  needs  of  most  practitio¬ 
ners.  Most  compelling,  however,  was  a  strong 
consensus  from  the  mini-symposium  partici¬ 
pants  that  the  technical  challenges  for  DIS  will 
be  rivaled  by  the  management  challenges  in 
developing  and  using  Distributed  Interactive 
Simulations. 
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CHAPTER  1 
INTRODUCTION 


DISTRIBUTED  INTERACTIVE  SIMU¬ 
LATION  (DIS) 

Distributed  Interactive  Simulation  (DIS) 
represents  a  new  and  evolving  area  spawned 
from  the  Defense  Advanced  Research  Projects 
Agency  (DARPA)  demonstrated  simulation 
networking  (SIMNET)  technology.  DIS 
encompasses  the  concept  of  multiple  simula¬ 
tion  components  provided  by  a  variety  of 
simulators  located  at  multiple  locations  which 
all  work  together  through  central  architectures 
and  standards  to  provide  a  common  synthetic 
battlefield  environment. 

THE  DIS  CONCEPT 

Combined  arms  teams  must  exercise  their 
tactics  and  doctrine,  sophisticated  communica¬ 
tions,  and  targeting  and  hand-off  systems,  but 
there  are  few  locations  available  for  such 
exercises.  Furthermore,  high  costs,  environ¬ 
mental  policies,  and  safety  concerns  restrict 
how  much  of  the  desired  combined  arms 
training  can  feasibly  be  undertaken.  To  com¬ 
pensate,  in  the  1980s,  DARPA  and  the  U.S. 
Army  initiated  a  joint  program  of  research  for 
real  time,  large  scale,  human-in-the-loop 
SEMulation  NET  working  or  "SIMNET." 

Intended  for  training  combined  arms  teams 
potentially  consisting  of  army,  naval,  and  air 
forces,  this  program  has  evolved  into  the  DIS 
concept.  "Distributed"  alludes  to  the  geo¬ 
graphic  dispersion  via  networking  of  both  the 
participants  interacting  with  simulations  por¬ 
traying  a  single  "virtual"  battlefield  environ¬ 
ment,  and  the  computational  and  communica¬ 
tion  resources  supporting  the  simulation. 


"Interactive"  refers  to  simulator  operators 
performing  actions  within  a  common  simula¬ 
tion  environment  and  subjected  to  results  that 
would  likely  occur  in  a  real  battle.  "Simula¬ 
tion"  is  considered  to  include  interfaces  with 
computer  combat  models  and  human-in-the- 
loop  simulators. 

The  Department  of  Defense  (DoD)  has  recog¬ 
nized  that  DIS  has  potential  in  additional 
arenas  besides  training  and  has  established 
synthetic  environments  as  a  science  technology 
thrust.  Combat  developers  could  use  DIS  to 
develop  requirements  and  assess  new  doctrine 
and  tactics  for  current  and  future  systems. 
Materiel  developers  could  save  money  and 
reduce  development  risks  by  employing  DIS  to 
prototype  and  exercise  system  modifications  or 
designs.  Testers  can  use  DIS  to  plan  tests,  or 
as  a  source  of  supplemental  data  for  their 
evaluations.  Military  analysts  have  already 
exploited  DIS  to  study  actual  combat  by  re¬ 
creating  battles. 

A  primary  recommendation  of  the  1991  Army 
Science  Board  on  Army  simulation  strategy 
was  that  the  common  representation  of  the 
battlefield  could  and  should  be  developed  to 
support  the  full  community  of  users  including 
developers,  testers  and  trainers  and  includes 
"seamless"  interconnection  across  different 
methods  of  simulation.  An  important  attribute 
of  the  DIS  environment  relates  to  the  suitabil¬ 
ity  for  human  interaction  and  representation 
within  the  times  of  human  perception  and  at 
the  appropriate  level  of  resolution  above 
detailed  engineering  design  and  physical  phe¬ 
nomena  models  and  simulation. 
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OBJECTIVES 

The  overall  objective  of  this  mini-symposium 
was  to  explore  military  operations  research 
applications  of  DIS  technology  and  its  envi¬ 
ronments.  The  mini-symposium  provided  a 
forum  to  examine  DIS  technology  applications 
in  analysis,  test  and  evaluation,  and  training. 

GOALS 

•  Provide  a  learning  experience  for  the 
participants. 

•  Examine  the  utility  and  limitations  of 
DIS  environments. 

•  Develop  issues  in  the  use  of  synthetic 
environments  for  benchmarking  and  data 
collection. 

•  Explore  the  ramifications  of  humans 
interacting  in  DIS  environments. 

SCOPE 

MORS  conducted  a  three-day  mini-symposium 
beginning  on  29  September  1992.  The  mini¬ 
symposium  was  limited  to  an  UNCLASSI¬ 
FIED  discussion  of  the  concepts,  capabilities, 
and  application  of  DIS  and  synthetic  combat 
environments.  The  mini-symposium  was 
conducted  at  the  Radisson  Mark  Plaza  Hotel 
and  at  the  Institute  for  Defense  Analyses  in 
Alexandria,  VA. 

All  who  registered  to  attend  the  mini-sympo¬ 
sium  were  sent  read-ahead  information  to 
familiarize  them  with  the  DIS  concept.  The 
read-ahead  papers  were  not  intended  to  be 
restrictive  or  exhaustive. 

The  mini-symposium  was  preceded  by  a  tuto¬ 
rial  on  the  evening  of  28  September  1992. 
The  tutorial  covered  an  overview  of  DIS 
technologies  and  emerging  interoperability 
standards  and  protocols.  Dr.  Duncan  C. 


Miller,  BBN  Systems  and  Technology,  Inc., 
discussed  "SIMNET  Architecture:  An  Histori¬ 
cal  Overview"  and  Dr.  Bruce  McDonald, 
University  of  Central  Florida  presented  "Stan¬ 
dards  for  the  Interoperability  of  Defense  Simu¬ 
lations."  Symposium  participants  were  en¬ 
couraged  to  attend  the  tutorial  in  order  to 
obtain  useful  background  information  for 
subjects  to  be  discussed  during  the  mini-sym¬ 
posium. 

On  the  first  day,  the  mini-symposium  had  a 
general  session  with  opening  remarks  by  Mr. 
E.  B.  Vandiver,  MORS  President,  an  orienta¬ 
tion  by  Dr.  Henry  C.  Dubin,  Mini-symposium 
Chairperson,  and  a  panel  discussion  by  speak¬ 
ers  (Dr.  Duncan  Miller,  BBN  Labs;  Dr.  Phil 
Dickinson,  E-Systems;  Dr.  Monti  D.  Callero, 
RAND;  and  Mr.  Edward  C.  Brady,  Strategic 
Perspectives  was  the  session  chair). 

The  morning  general  session  was  followed  by 
a  lunch  with  a  keynote  speech  by  Mr.  John 
Hamre,  professional  staffer  for  the  U.S.  Senate 
Armed  Services  Committee. 

The  afternoon  general  session  on  the  first  day 
focused  on  the  use  of  DIS  technology  to  date. 
Four  "seasoned  veterans"  discussed  their 
experiences  using  DIS  to:  evaluate  new  tech¬ 
nologies;  develop  system  requirements;  de¬ 
velop  tactics,  techniques,  and  procedures;  and 
testing.  Each  of  these  experienced  users 
presented  problems  encountered  and  lessons 
learned,  his/her  view  of  the  capabilities  and 
limitations  of  DIS  technology,  and  recommen¬ 
dations/plans  for  its  future  use.  Mr.  Richard 
E.  Garvey  was  the  session  chair,  and  papers 
were  presented  by  the  following  four  "sea¬ 
soned  veterans"  (Dr.  Barbara  A.  Black,  ARI 
Field  Unit,  Fort  Knox;  LTC  Keith  M.  Moore, 
Office  of  the  Assistant  Secretary  of  the  Army 
(Research,  Development  and  Acquisition) 
(OASA(RDA));  CAPT  H.  C.  Kaler,  Naval  Sea 
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Systems  Command;  and  Mr.  John  V.  Meier, 
Los  Alamos  National  Laboratory). 

On  the  second  day,  working  groups  focused 
on  examining  particular  areas  of  interest  in 
greater  detail.  The  working  groups  were 
chaired  by  senior  individuals  with  particular 
interest  in  the  field.  The  working  group  ses¬ 
sions  focused  on  discussions  designed  to 
inform,  generate  interest  in,  and  improve  the 
understanding  of  DIS  and  synthetic  combat 
environments  in  the  community.  Each  work¬ 
ing  group  identified  uses,  limitations  and 
needed  enhancements  of  DIS  to  support  their 
specific  needs.  Each  group  was  encouraged  to 
address  human  performance  and  behavioral 


least  one  of  the  working  group  topics.  This 
goal  was  achieved  with  a  wealth  of  back¬ 
grounds  covering  disciplines  including  engi¬ 
neering,  systems  design,  testing,  force  struc¬ 
turing,  tactics/doctrine  development,  support 
analysis  for  all  components  of  the  U.S.  Armed 
Forces,  national  laboratories,  federally  funded 
research  and  development  centers,  and  private 
contractual  firms.  Additionally,  participants 
were  to  come  prepared  to  play  an  active  role 
as  recorder,  moderator,  or  discussion  leader. 

WORKING  GROUPS 

TABLE  1-1.  ATTENDANCE  BY 
AFFILIATION 


The  mini-symposium  concluded  on  the  morn¬ 
ing  of  the  third  day  with  reports  from  the 
working  groups.  The  working  group  chairs 
reported  the  findings  of  their  groups  and 
identified  any  possible  follow-on  efforts  appro¬ 
priate  for  MORS  support. 

PARTICIPATION 

Attendance  was  not  restricted.  The  goal  was 
to  get  a  mix  of  people  with  various  levels  of 
DIS  experience,  but  more  importantly  each 
individual  should  be  currently  involved  with  at 


WORKING  GROUPS 

There  were  10  working  groups.  The  working 
group  titles,  the  primary  questions  that  they 
examined,  their  chairs  and  co-chairs,  and  their 
approximate  number  of  participants  follow. 

Working  Group  1.  Military  Analysis.  How 

can  DIS  supplement  or  extend  existing  military 
analysis  techniques?  How  does  DIS  relate  to 
existing  military  analysis  techniques?  Chair: 
Mr.  Kent  Pickett,  Director,  Model  Directorate, 
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U.S.  Army  Training  and  Doctrine  Command 
(TRADOC)  Analysis  Command  -  Operations 
Analysis  Center.  Co-Chair:  Mr.  Wallace 
Chandler,  Acting- Assistant  Director,  Research 
and  Analysis  Support  Directorate,  U.S.  Army 
Concepts  Analysis  Agency.  Approximate 
number  of  participants:  29. 

Working  Group  2.  Test  and  Evaluation 
(T&E).  How  can  DIS  be  appropriately 
applied  to  support  or  supplement  T&E? 
Chair:  COL  Bernard  Ferguson,  Executive 
Assistant  for  the  Director  for  Test  and  Evalua¬ 
tion,  Office  of  the  Under  Secretary  of  Defense, 
Acquisition.  Co-Chair:  Dr.  Adelia  Ritchie, 
Manager,  Information  Systems  Division, 
Science  Applications  International  Corpora¬ 
tion.  Approximate  number  of  participants: 
31. 

Working  Group  3.  Operations  Planning 
and  Rehearsal.  How  can  DIS  be  applied  to 
planning  and  rehearsing  for  military  opera¬ 
tions?  Chair:  Commander  Dennis  McBride, 
Program  Manager,  Defense  Advanced  Re¬ 
search  Projects  Agency.  Co-Chair:  CAPT 
Bruce  McClure,  Chief,  Modeling  and  Analysis 
Section,  Office  of  Naval  Operations.  Approxi¬ 
mate  number  of  participants:  34 

Working  Group  4.  Logistics,  Mobilization 
and  Sustainment.  How  will  DIS  incorporate 
means  to  properly  address  logistics,  mobiliza¬ 
tion,  and  sustainment?  How  can  DIS  be  used 
to  address  logistical,  mobilization,  and  sustain¬ 
ment  issues?  Chair:  Dr.  Lisa  Sokol,  MRJ, 
Inc.  Co-Chair:  Mr.  AI  Irwin,  Science  Appli¬ 
cations  International  Corporation.  Approxi¬ 
mate  number  of  participants:  27. 

Working  Group  5.  Force  Developments. 

What  types  of  DIS  can  contribute  to  force 
development  analysis  and  how?  Chair:  Dr. 
Darrell  Collier,  Director,  TRADOC  Analysis 


Command,  White  Sands  Missile  Range.  Ap¬ 
proximate  number  of  participants:  30. 

Working  Group  6.  Combat  Developments. 

How  can  DIS  be  used  to  develop  the  most 
cost-effective  solutions  to  mission  needs? 
Chair:  Mr.  Mike  Bauman,  Acting  Director, 
U.S.  Army  TRADOC  Analysis  Command. 
Co-Chair:  MAI  Jeffrey  Wilkinson,  Chief, 
Close  Combat  Test  Bed,  USAAC  Directorate 
of  Combat  Development  (DCD).  Approxi¬ 
mate  number  of  participants:  23. 

Working  Group  7.  Training  and  Readi¬ 
ness.  How  can  DIS  be  used  for  cost  effective 
training  and  readiness?  Chair:  Mr.  Kenneth 
Lavoie,  Technical  Director,  Air  Force  War¬ 
gaming  Center.  Co-Chair:  Colonel  Steven  S. 
Overstreet,  Acting  Project  Manager  of  Close 
Combat  Tactical  Trainer,  Simulation,  Training 
and  Instrumentation  Command.  Dr.  Stanley 
Halpin,  Chief,  Field  Unit,  Army  Research 
Institute.  Approximate  number  of  partici¬ 
pants:  26. 

Working  Group  8.  Requirements  Develop¬ 
ment  and  Definition.  How  can  DIS  be  used 
to  identify,  develop,  and  validate  require¬ 
ments?  Chair:  Dr.  James  Metzger,  Operations 
Research  Analyst,  Office  of  Assistant  Secre¬ 
tary  of  Defense  (Program  Analysis  &  Evalua¬ 
tion).  Co-Chair:  Colonel  Gilbert  M.F. 
Brauch,  Jr.,  Chief,  U.S.  Army  Model  and 
Simulation  Management  Office,  U.S.  Army 
Model  Improvement  and  Study  Management 
Agency,  Office  of  the  Deputy  Under  Secretary 
of  the  Army-Operations  Research  (DUSA- 
OR),  Headquarters,  Department  of  the  Army 
(HQDA).  Approximate  number  of  partici¬ 
pants:  34. 

Working  Group  9.  Simulation  Prototyping 
to  Support  Acquisition.  How  can  proto¬ 
typing  using  DIS  environments  support  the 
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materiel  acquisition  process?  What  do  we 
mean  by  rapid  prototyping  of  simulations? 
Chair:  Mr.  Dick  Garvey,  Director, 

Leavenworth  Operations,  BDM  International. 
Co-Chair:  Dr.  RonHofer,  Technical  Director, 
Simulation  Training  and  Instrumentation 
Command.  Approximate  number  of  partici¬ 
pants:  33. 


Working  Group  10.  Cost/Benefit/Risk  of 
DIS.  What  are  the  costs,  benefits,  and  risks 
associated  with  DIS  applications  to  training, 
military  analysis,  force  development,  combat 
development,  and  test  and  evaluation?  Chair: 
Mr.  Ed  Brady,  Consultant,  Strategic  Perspec¬ 
tives.  Co-Chair:  Dr.  Peter  Cherry,  Vice 
President,  Vector  Research  Inc.  Approximate 
number  of  participants:  28. 
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CHAPTER  2 
WORKING  GROUP  1 
MILITARY  ANALYSIS 

Kent  Pickett 


BACKGROUND/OBJECTTVES 

The  objective  of  Working  Group  1  was  to 
explore  the  uses  of  Distributed  Interactive 
Simulations  (DIS)  in  support  of  military 
analysis.  The  group  consisted  of  29  mem¬ 
bers  with  varying  backgrounds  in  military 
analysis.  These  backgrounds  included  work¬ 
ing  in  weapon  engineering  design,  weapon 
testing,  force  structuring,  development  of 
tactics/doctrine  and  theater  support  analysis, 
to  cite  only  a  few.  Members  of  the  group 
also  represented  organizations  of  all  compo¬ 
nents  of  the  U.S.  Armed  Forces.  Several 
nongovernmental  organizations  present  in¬ 
cluded  a  national  laboratory,  a  federally 
funded  research  and  development  center,  and 
private  military  contractual  firms. 

Early  in  the  session,  two  subgroups  were 
established.  The  subgroups  addressed  the 
following  topics: 

•  Subgroup  1  reviewed  those  areas  of 
current  DIS  technology  which  could  be 
applied  to  analytic  tasks.  The  group 
chose  to  limit  these  discussions  to 
technology  that  could  be  reasonably 
expected  to  mature  in  the  next  five 
years.  Included  in  these  discussions 
was  the  identification  of  those  areas 
where  DIS  was  clearly  not  applicable. 
Subgroup  1  closed  its  discussions  with 
several  recommendations  for  changes 
and  additions  to  the  DIS  system  to 
make  the  architecture  more  usable  for 
military  analysis. 


•  Subgroup  2  focused  on  the  procedures 
to  verify  and  validate  DIS  based  mod¬ 
els  and  architectures  for  potential  use 
in  military  analysis.  The  subgroup 
found  that  verification  and  validation  in 
a  distributed,  multi-resolution  environ¬ 
ment  provides  a  different  set  of  chal¬ 
lenges  than  those  experienced  in  vali¬ 
dating  conventional  models  and  simula¬ 
tions.  Recommendation  for  research 
topics  to  establish  a  basis  for  V&V 
activities  in  DIS  resulted  from  the 
discussions  of  this  subgroup. 

CURRENT  ANALYTIC  USES  FOR  DIS 
AND  RECOMMENDATIONS  FOR  IM¬ 
PROVEMENTS. 

The  general  consensus  of  Subgroup  1  was 
that  the  DIS  environment  provides  a  valuable 
laboratory  in  which  the  analytic  community 
can  learn  more  about  human  interaction  with 
new  and  existing  battlefield  systems  than  has 
heretofore  been  possible.  This  knowledge 
can  be  used  to  support  both  early  develop¬ 
ment  of  the  new  system  concept  in  terms  of 
how  to  use  it  effectively  on  the  battlefield 
and  how  the  system  can  be  integrated  into  the 
existing  force.  Further,  the  subgroup  be¬ 
lieved  that  DIS  can  be  effectively  used  in  the 
areas  of  battle  planning,  tactics/doctrinal 
development  and  force  structuring  for  cur¬ 
rently  existing  systems. 

The  subgroup,  however,  did  not  believe  that 
DIS  can  be  used  effectively  for  investigating 
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the  basic  physics  underlying  the  performance 
envelope  of  new  weapon  systems.  In  short, 
the  DIS  model  can  be  no  better  than  our 
understanding  of  the  physical  phenomena 
supporting  the  system  and  the  virtual  envi¬ 
ronment  of  DIS  has  only  limited  capabilities 
in  expanding  our  understanding  of  the  princi¬ 
ples.  High  resolution  engineering  models  are 
still  needed  when  doing  tradeoffs  in  weapon 
envelope  performance.  DIS  can  give  us 
insights  on  how  human  beings  can  best  ex¬ 
ploit  these  performance  changes  in  battle. 

In  the  area  of  development  for  new  systems, 
the  group  found  the  virtual  environment 
technologies  of  SIMNET/SAFOR  and  DIS 
have  a  high  potential  for  investigating  the 
impact  of  human  factors  early  in  the  develop¬ 
ment  of  a  system.  These  types  of  analysis 
include: 

•  Front  End  Analysis  describing  the 
mission,  requirements  and  the  opera¬ 
tional  concept  of  the  new  system. 

•  Operation  Mode  Summary/  Mission 
Profile  analysis  describing  how  the 
new  system  will  be  used  and  how  often 
the  system  will  find  itself  in  particular 
modes  of  operation. 

•  Organizational  relationships  between 
the  new  systems  and  current  systems. 
This  is  a  further  refinement  of  the 
Organizational  concept  and  will  help 
lay  the  basis  for  force  structuring  anal¬ 
ysis  with  the  new  system. 

The  key  advantage  in  the  use  of  the 
SIMNET/  SAFOR  virtual  environment  in  the 
early  development  of  a  system  is  that  it  gives 
the  military  analyst  an  insight  into  how 
soldiers  will  use  the  proposed  weapon  sys¬ 
tem.  Current  analysis  is  primarily  based  on 
how  soldiers  use  systems  based  on  similar 


technologies  to  those  of  the  new  system.  But 
in  cases  where  the  technology  is  radically 
new  (e.g.,  the  use  of  a  laser  system  as  the 
main  armament  as  opposed  to  a  conventional 
munitions  or  missiles)  this  environment  gives 
us  an  early,  objective  look  at  how  to  employ 
the  new  system  without  actually  going  to  the 
expense  of  building  it. 

The  Subgroup  believes  that  SIMNET/ 
SAFOR  also  has  the  potential  to  be  an  effec¬ 
tive  tool  in  the  area  of  Mission  Planning. 
The  virtual  environment  gives  the  com¬ 
mander  the  ability  to  "walk  the  battlefield" 
before  the  actual  battle.  It  requires  improve¬ 
ment  for  this  to  become  a  common  use  of  the 
system.  Over  the  next  five  years,  however, 
it  was  believed  that  improved  graphics  in 
SIMNET  will  become  a  reality.  In  this  case 
the  SIMNET  stealth  capability  might  well 
support  the  required  terrain  analysis  defining 
likely  positions  for  friendly  and  enemy  troops 
and  even  support  tactics  development  through 
virtual  battle  rehearsal. 

If  SIMNET/SAFOR  is  to  realize  its  potential 
as  a  usable  tool  in  the  analytic  community, 
certain  improvements  will  be  necessary.  The 
Subgroup  listed  the  following  upgrades  as 
basic  improvements  which  should  be  accom¬ 
plished  to  insure  the  usefulness  of  the  system 
throughout  the  analytic  community: 

•  A  review  of  existing  SIMNET/ 
SAFOR  algorithms,  databases  and 
methodologies  must  be  conducted.  In 
short,  the  basic  representation  of  sys¬ 
tem/battle  processes  must  be  verified 
and  validated.  This  is  not  to  imply 
that  work  in  the  area  has  not  already 
begun.  Several  successful  efforts  have 
been  sponsored  by  DARPA  and 
STRICOM  in  an  effort  to  make 
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SIMNET  a  more  realistic,  respon¬ 
sive  training  tool.  The  implication 
here  is  that  these  V&V  efforts  must 
continue  and  that  the  analytical 
community  needs  to  actively  share 
the  responsibilities  in  these  re¬ 
views. 

The  environmental  representations  in 
SIMNET/SAFOR  need  upgrading. 
Much  of  the  developmental  work  in 
new  combat  systems  depends  on  new 
sensors.  The  environment  strongly 
affects  how  well  or  poorly  these  sen¬ 
sors  perform.  SIMNET/SAFOR  must 
be  able  to  fairly  represent  the  attenuat¬ 
ing  effects  of  both  the  natural  and  man¬ 
made  environment.  This  includes  both 
physical  obscurants  and  electromag¬ 
netic  jamming  affecting  sensors  and 
seekers.  The  subgroup  recognized  this 
as  a  high  priority  in  current  DIS/ 
SIMNET  development  and  simply 
wishes  to  reinforce  DAEPA/ 
STRICOM's  efforts. 

The  SAFOR/Computer  Generated 
Forces  (CGF)  should  have  the  facility 
to  run  in  a  repeatable/systemic  mode 
with  intelligent  responses  of  vehicles  to 
dynamic  battle  conditions.  The  analyt¬ 
ical  community  commonly  uses  a  sen¬ 
sitivity  based  methodology  to  establish 
cause  and  effect  between  new  weapon 
systems  and  battle  results.  In  this  type 
of  analysis,  a  "base  case"  is  established 
using  the  combat  model  of  choice. 
The  base  case  is  carefully  set  up  with 
valid  battle  positions  for  units  perform¬ 
ing  particular  missions.  The  base  case 
is  reviewed  until  it  appears  reasonable 
under  the  threat,  battlefield,  and 
friendly  force  conditions.  The  battle  is 
run  and  the  results  are  reviewed.  A 
modification  representing  the  new 


weapon's  impact  on  the  force  is  then 
made  to  the  base  case.  This  "modifica¬ 
tion"  may  be  as  simple  as  changing  the 
performance  envelope  of  an  improved 
system  or  it  may  be  as  complex  as 
restructuring  the  force  with  an  associ¬ 
ated  change  in  mission  and  weapon 
positions.  In  either  case,  the  appropri¬ 
ate  changes  are  made  and  the  battle 
rerun  under  the  new  conditions.  Battle 
outcomes  are  then  compared  to  the 
base  case.  The  point  is  simply  that  if 
SAFOR  is  to  become  the  high  resolu¬ 
tion  model  of  analytic  choice,  there  is 
a  need  for  a  single  analyst  to  run  the 
model  without  simulators  or  a  staff  of 
gamers  guiding  the  individual  vehicles 
as  they  move  through  the  battle.  This 
is  not  meant  to  imply  that  the  SAFOR 
would  be  used  purely  in  an  "un¬ 
attended"  mode.  Most  certainly,  a 
reasonable  analyst  would  carefully 
review  the  new  battles  for  validity. 
There  must  be  a  capability  for  the 
SAFOR  to  run  in  repeatable,  interrupt 
capable,  stand-alone  mode  when  given 
a  sound  tactical  plan  at  the  division/- 
brigade/  battalion  level. 

VERIFICATION  AND  VALIDATION 
FOR  MILITARY  ANALYSIS  IN  A  DIS¬ 
TRIBUTED  SIMULATION  ENVIRON¬ 
MENT 

Subgroup  2  of  the  Military  Analysis  Working 
Group  developed  their  discussions  from 
consideration  of  two  base  points: 

•  DIS  is  in  fact  a  new  simulation  envi¬ 
ronment  which  places  different  de¬ 
mands  on  the  W&A  process  than 
those  associated  with  and  documented 
for  independent  "constructive"  simula- 
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tion  models. 

A  particular  DIS  environment  (net¬ 
work,  nodes,  communication  proto¬ 
cols,  hardware  and  software  compo¬ 
nent,  etc.)  does  not  acquire  or  inherit  a 
degree  of  quality  assurance  from  the 
fact  that  some  or  all  of  its  component 
parts  have  been  subjected  independ¬ 
ently  to  VV&A.  In  short,  verification 
and  accreditation  of  the  parts  does  not 
imply  verification  and  accreditation  of 
the  whole  system.  The  discussion  in 
the  group  focused  more  on  verification 
and  accreditation  and  less  on  validation 
issues. 

The  important  aspects  of  the  DIS  envi¬ 
ronment  affecting  the  VV&A  problem 
are  its  distributed  nature  and  its  hetero¬ 
geneity  of  resolution.  Its  interactive 
quality,  also  an  important  aspect  with 
respect  to  W&A,  was  not  considered 
as  prominently  since  there  is  not  a 
consensus  on  whether  man-in-the-loop 
is  a  necessary  condition  to  a  DIS  envi¬ 
ronment. 

Several  problems  impact  an  effective 
VV&A  effort  of  a  DIS  based  system 
being  used  for  analysis.  The  system 
may  be  so  geographically  distributed 
that  not  all  components  are  available 
for  review.  It  may  be  distributed 
across  multiple  hardware  platforms 
with  various  modes  of  communication 
among  them.  Simulators  and  closed 
simulation  operating  at  different  levels 
of  resolution,  aggregation,  or  fidelity 
may  all  be  "interoperating"  on  the 
network.  Models  supporting  a  particu¬ 
lar  DIS  application  may  produce  out¬ 
puts  which  are  deterministic,  or  repre¬ 
sent  single  or  multiple  instances  of  a 
stochastic  process.  A  transition  from 
one  hardware  platform,  type  of  model 


or  level  of  aggregation  to  another  in¬ 
troduces  the  possibility  of  misinterpre¬ 
tation  or  loss  of  information.  Each  of 
these  "boundary  crossing"  situation 
must  be  examined  and  verified  as  part 
of  the  accreditation  process. 

The  group  felt  the  situation  called  for  the 
development  of  new  procedures  and  new 
theory  to  meet  these  demanding  VV&A 
conditions  of  the  DIS  environment.  Several 
aspects  of  this  required  development  were 
identified  as  follows: 

•  Methods  for  verifying  the  "boundary 
crossing"  interfaces  between  compo¬ 
nents  of  the  DIS  structure  being  used 
for  analysis. 

•  Requirements  for  certifying  personnel 
playing  a  man-in-the-loop  role. 

•  Procedures  for  calibration  of  the  sys¬ 
tem  components  with  different  levels 
of  aggregation. 

•  Experimental  design  methods  to  sup¬ 
port  interoperation  of  statistically  dis¬ 
similar  simulations. 

•  Procedures  to  evaluate  the  physical 
(electronic)  limitation  or  restriction  of 
a  particular  network  architecture. 

•  More  comprehensive  documentation 
for  components  which  cannot  be  exam¬ 
ined  first  hand  by  a  DIS  user. 

•  Better  data  collection  and  data  manage¬ 
ment  techniques  which  allow  increased 
visibility  and  monitoring  of  system 
interfaces. 

In  summary,  the  subgroup's  recommenda¬ 
tions  were  consolidated  under  four  principal 
points: 

•  Seek  a  way  to  encourage  and  sponsor 
the  development  of  theory  and  method- 
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ology  to  address  the  issues  cited 
above. 

•  Promote  the  documentation  of  such 
research  efforts  through  journal  arti¬ 
cles,  issue  papers  or  position  papers 
with  the  ultimate  goal  of  the  develop¬ 
ment  of  primers  or  handbooks  to  in¬ 
struct  the  analyst  DIS  user  in  how  to 
accredit  a  system  for  particular  analytic 
applications 


•  Start  collecting  data  on  system  inter¬ 
faces  and  issues  associated  therewith 
using  current,  not  necessarily  analytic, 
application  of  DIS  technology. 

•  Identify  and  design  experiments  and 
test  cases  for  DIS  applications  which 
can  furnish  information  to  support  the 
exploration  of  the  issues  cited  above. 
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CHAPTER  3 
WORKING  GROUP  2 
TEST  AND  EVALUATION  (T&E) 
Colonel  Bernard  Ferguson 


PURPOSE 

To  examine  Distributed  Interactive  Simula¬ 
tion  (DIS)  as  it  might  be  used  to  support  Test 
and  Evaluation,  identifying  uses,  limitations, 
and  enhancements  required. 

BACKGROUND 

DIS  has  had  very  limited  use  in  Test  and 
Evaluation  (T&E)  to  date.  The  Working 
Group  heard  a  briefing  from  one  member 
who  was  involved  with  the  testing  of  NLOS 
using  DIS.  Based  upon  that  briefing,  upon 
the  demonstration  at  the  IDA  Simulation 
Facility,  and  on  the  group's  expertise,  the 
Working  Group  proceeded  to  examine  the 
use  of  DIS  to  support  T&E,  focusing  on  the 
differences  between  DIS  and  stand-alone 
simulations  of  similar  size  or  complexity. 

The  Working  Group  discussed  when  to  use 
and  when  not  to  use  DIS  in  T&E.  They 
developed  several  key  limitations  to  DIS 
applications,  most  of  which,  if  resolved, 
would  make  DIS  applications  in  T&E  more 
feasible.  Key  research  topics,  where  more 
consideration  and  deliberation  will  be  re¬ 
quired,  were  discussed  and  conclusions  and 
recommendations  were  developed,  as  dis¬ 
cussed  in  the  following  paragraphs. 

TOPICAL  DISCUSSION 

The  T&E  Working  Group  was  charged  with 
proposing  potential  applications  of  DIS  in 
test  and  evaluation.  In  addition  to  those 


instances  where  DIS  should  be  considered 
applicable,  the  Working  Group  developed  a 
set  of  circumstances  where  DIS  should  not  be 
considered  for  T&E  applications. 

When  to  Consider  DIS  for  T&E  Applica¬ 
tions.  In  theory,  DIS  could  be  applied 
throughout  the  life  cycle  of  a  weapon  sys¬ 
tem’s  acquisition  process,  e.g.,  in  require¬ 
ments  generation,  concepts  analysis,  Opera¬ 
tional  Utility  Evaluations  (OUEs),  Early 
Operational  Assessments  (EOAs),  Initial  and 
Follow-on  Operational  Test  and  Evaluation 
(IOT&E  and  FOT&E),  etc.  Of  particular 
interest  to  the  T&E  community  is  that  a 
consistent  DIS  application  could  provide  a 
means  for  linking  Cost  and  Operational 
Effectiveness  Analyses  (COEAs)  to  Test  and 
Evaluation  Master  Plans  (TEMPs).  Also, 
applications  in  FOT&E  could  provide  front 
end  screening  or  a  view  of  the  next  genera¬ 
tion  system  or  mission  requirements. 

DIS  could  be  used  to  help  identify  test  issues, 
to  train  participants,  and  to  refine  tactics  and 
doctrine.  The  Working  Group  believes  that 
DIS  may  allow  practical  tactics  to  be  exer¬ 
cised.  For  a  given  system,  test  planners 
must  identify  the  issues  to  be  addressed  by 
DIS.  The  advantage  is  that  this  process  may 
force  more  advance  thinking  and  planning  for 
test  resources,  including  requirements  for 
DIS  which  is  itself  a  test  resource  in  this 
case.  Test  planning  must  be  conducted  early, 
as  early  as  Milestone  0,  in  order  for  the 
projected  capabilities  to  be  available  at  test 
time.  This  process  will  be  facilitated  if  the 
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acquisition  strategy  invokes  DIS  consistently 
throughout  all  milestones. 

Like  "modeling  and  simulation,"  DIS  could 
be  invoked  when  constraints  require  alterna¬ 
tives  to  testing,  e.g.,  in  the  case  of  nuclear, 
biological,  chemical,  and  space  systems. 
Constraints  that  force  testers  to  use  alterna¬ 
tive  tools  are  generally  in  the  categories  of 
environmental,  safety,  political,  electromag¬ 
netic  emanations,  cost,  and  the  like. 

Other  potential  applications  include  Multi- 
Service  procurement  (combined  arms,  many 
players),  Joint  Tests  (e.g.,  JSTARS, 
JLOTS),  and  highly  concurrent  systems, 
usually  the  high  cost  items,  when  sufficient 
numbers  of  the  system  are  not  ready  in  time 
to  conduct  tactical  unit  testing  prior  to  a 
production  decision. 

A  large  proportion  of  major  system  testing  is 
classified  at  the  Secret  level  or  even  higher. 
The  net  must  be  secure,  with  all  players  and 
sites  being  cleared  as  well. 

When  Not  to  Use  DIS  in  T&E.  Technology 
limitations  and  common  sense  are  the  guide¬ 
lines  in  this  case.  Latency  problems  preclude 
applications  for  highly  mobile  systems.  DIS 
is  not  needed  if  the  system  under  test  does 
not  need  to  interact  with  other  systems  or  if 
the  system  is  easy  and  inexpensive  to  test 
normally.  Engineering  testing,  "shake  and 
bake,"  does  not  require  DIS,  nor  could  the 
working  group  find  any  potential  applications 
for  this  phase  of  testing.  DIS  is  not  for 
testing  of  Major  Automated  Information 
Systems  (MAIS)  nor  any  other  information 
management  systems.  Above  all,  when  the 
physics  of  the  system  under  test  is  not  well 
understood  DIS  must  be  used  either  with 
caution  or  not  at  all. 


LIMITATIONS 

An  extensive  list  of  limitations  exists,  several 
of  which  are  technical,  bureaucratic,  and 
administrative,  that  could  preclude  DIS 
applications  in  T&E. 

•  Security  -  DIS  should  not  be  applied  in 
T&E  until  multilevel  security  issues  are 
resolved  (networks  must  be  classified 
for  most  applications). 

•  Accuracy  of  representation  of  sensors 
&  countermeasures  (IR,  laser,  etc.) 

•  Dynamic  environments  (terrain,  clutter) 

•  Protocol  limitations  (latency,  flexibil¬ 
ity,  volume) 

•  Network  management  -  Who's  in 
charge? 

•  Consistency  between  SAFOR  and  man- 
in-the-loop 

•  Logistics  may  be  more  difficult  to  play. 

•  Test  control  may  be  more  difficult. 
DIS  requires  increased  coordination, 
planning  time,  etc.  More  players  are 
involved,  more  difficult  to  budget, 
schedule.  But,  more  in-depth 

test  planning  may  be  possible. 

•  Data  collection,  reduction,  and  han¬ 
dling  may  be  more  difficult. 

•  W&A  issues  are  more  complex. 

•  DIS  requires  distribution  and  mainte¬ 
nance  of  common  data  bases. 

ISSUES 

Protocols  When  considering  protocols  in 
DIS,  one  is  considering  protocol  data  units 
(PDUs)  used  to  transmit  information  on  the 
network.  The  Working  Group  used  Version 
1.0  in  considering  PDU  capabilities,  because 
this  is  the  only  version  close  to  being  ac¬ 
cepted  as  a  standard.  The  working  group 
discussed  the  following  problem  areas  and 
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possible  solutions: 

LATENCY.  This  is  the  delay  experienced 
between  an  entity  state  change  and  the  time 
at  which  other  entities  on  the  network  be¬ 
come  aware  of  that  change.  This  delay 
occurs  because  of  transmission  time,  PDU 
processing  time,  and  other  factors.  This 
problem  can  never  be  completely  overcome, 
but  it  might  be  significantly  reduced  by  use 
of  fibre  optics  or  token  rings  —  devices  and 
procedures  allowing  two  particular  entities  to 
transmit  data  direct  for  a  very  limited  time  at 
very  high  exchange  rates.  Particular  entities 
might  be  assigned  lower  update  thresholds  — 
the  amount  of  state  change  requiring  a  state 
change  PDU  be  transmitted.  Update  algo¬ 
rithms  could  be  changed  for  critical  systems 
to  speed  the  update  rather  than  focus  on 
presentation  graphics  movement.  One  could 
use  these  techniques  when  critical  entity  state 
changes  needed  to  be  transmitted  to  another 
entity. 

FLEXIBLE  PDUs.  To  gather  data  not 
available  from  standard  PDUs,  testers  need  a 
special  request  PDU  or  one  with  blank  slots. 
These  would  give  the  capability  to  gather 
such  data. 

NETWORK  PDU  VOLUME  CAPABILITY. 
The  volume  capacity  of  current  networks 
limits  the  number  of  PDUs  that  the  system 
can  handle  at  any  one  time.  This  in  turn 
limits  the  number  of  entities  that  can  be 
represented  and  the  amount  of  data  that  can 
be  transmitted.  This  problem  is  directly 
related  to  the  bandwidth  of  the  network. 
There  are  techniques,  however,  that  can 
help:  data  compression  techniques,  combin¬ 
ing  like  PDUs,  multicast,  and  smart  gate¬ 
ways.  These  methods  of  handling  PDU 
transmissions  have  the  potential  of  minimiz¬ 


ing  the  associated  problems. 

TEST  PLANNING.  Testers  should  plan  for 
the  early  use  of  DIS.  They  should  plan  for 
the  capability  projected  to  be  available  at  the 
time  they  plan  to  execute  their  test.  This  will 
require  much  coordination  with  the  "DIS  and 
Defense  Simulation  Internet  (DSI)  managing 
agencies"  very  early  in  the  acquisition  cycle. 
From  this  respect,  the  tester's  planning 
would  be  much  easier  if  the  system  acquisi¬ 
tion  strategy  included  the  use  of  DIS/DSI 
throughout  —  requirements  definition,  con¬ 
cept  trade  off  analysis,  developmental  test¬ 
ing,  and  operational  testing. 

VERSION  COMPATIBILITY.  Future 
versions  of  DIS  should  be  backwards  com¬ 
patible.  This  will  allow  testers  to  make  use 
of  new  capabilities  in  future  versions  with 
minimum  adjustment  from  previous  testing. 

SYSTEM  MONITOR.  When  a  critical  node 
drops  off  of  the  net  due  to  a  system  failure, 
the  tester  needs  to  know  so  he/she  can  stop 
the  test  until  that  node  is  brought  back  on 
line.  Other  failures  may  be  of  equal  impor¬ 
tance.  Identifying  such  failures  could  be 
accomplished  through  the  use  of  smart  gate¬ 
ways  and  special  PDUs,  generated  by  the 
failure. 

Who  Is  in  Charge?  DIS/DSI  needs  a  net¬ 
work  manager  to  perform  functions  such  as 
scheduling,  assuring  asset  readiness,  coordi¬ 
nation  of  user  system  needs,  and  planning 
enhancements  —  integration  and  budgeting. 
In  the  near  term,  this  network  manager 
should  be  advertising  the  system  --  letting 
installations  know  the  capability  that  will  be 
available  so  users  can  plan  and  budget  for 
nodes.  In  the  future,  the  network  manager 
would  serve  as  the  single  point  of  contact  for 
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test  planners. 

Each  installation  with  a  node  will  need  a  site 
manager.  This  person  would  schedule  use  of 
the  node  and  use  of  local  simulations  and 
simulators  which  the  user  needs  to  run  a  test 
or  other  application. 

Timing.  Timing  on  the  network  is  an  all 
important  issue;  especially  for  events  such  as 
the  missile  end  game  and  aircraft  evasive 
maneuvers.  Just  as  important,  all  entities 
need  to  be  running  on  the  same  reference 
time.  The  latter  issue  will  be  resolved 
through  the  use  of  the  Global  Positioning 
System  to  provide  the  reference  time  for  all 
entities.  The  former  issue  partially  revolves 
around  latency  which  we  discussed  earlier. 
Compatible  processing  capability  between 
nodes  can  affect  timing.  If  one  or  more 
nodes  have  a  lower  processing  capability,  the 
other  nodes  can  overload  those  with  limited 
capability  and  eventually  jam  those  nodes. 
Once  that  happens,  the  lower  capability 
nodes  will  drop  off  the  system  and  be  lost. 
Therefore,  the  nodes  should  have  compatible 
capabilities. 

Linkage  -  Live  Systems  to  Virtual  Envi¬ 
ronments.  Use  of  an  actual  weapon  system 
on  the  network  —  vs.  use  of  simulators  — 
would  enhance  fidelity  and  make  the  verifica¬ 
tion,  validation,  and  accreditation  process 
more  simple.  Weapon  systems  might  be 
linked  through  their  data  buses  —  not  a  cur¬ 
rent  capability.  The  desirability  of  doing  this 
is  certainly  system  and  application  depend¬ 
ent.  PDUs  would  have  to  be  developed  or 
modified  to  transmit  information  used  to 
simulate  the  system.  This  is  a  task  which 
requires  much  research,  but  one  that  may  pay 
great  dividends  in  terms  of  credibility. 


Viruses.  Viruses  transmitted  on  any  network 
are  potentially  catastrophic  -  and  no  less  so 
on  DIS.  Scanners  and  protective  techniques 
must  be  used.  DIS  also  opens  the  door  for 
false  PDUs.  Imagine  running  a  test  and  all 
of  a  sudden  a  ship  or  airplane  disappears! 
Smart  gateways  and  security  encryption  may 
overcome  these  potential  problems,  but  users 
should  be  aware  of  the  potential  risks. 

Reliability.  Several  testers  in  the  working 
group  who  have  previously  used  DIS/DSI 
experienced  reliability  problems.  Parts  of 
the  system  failed  during  a  test,  thus  stopping 
the  test  until  repairs  were  complete.  In  a 
large  scale  test  this  might  be  very  expensive 
in  terms  of  resource  availability,  data  lost,  or 
time  lost.  Testers  using  DIS/DSI  should  be 
aware  of  this  potential  and  consider  using 
back-up  capability  such  as  automatic  switch¬ 
ing  to  redundant  hardware  and  redundant 
transmission  links  for  critical  nodes.  Devel¬ 
opers  of  the  DIS  system  should  provide 
system  monitoring  so  a  person  monitoring 
the  test  could  be  advised  when  a  node  or  link 
fails,  thus  allowing  the  appropriate  actions  to 
be  taken. 

KEY  RESEARCH  TOPICS 

Topics  discussed  below  were  determined  to 
be  essential  issues  for  resolution  to  enhance 
the  T&E  community's  confidence  in  DIS. 

SAFOR  was  deemed  inadequate  to  support 
the  realism  required  by  the  T&E  community, 
its  critics,  and  its  overseers.  From  the  opera¬ 
tional  testing  perspective,  more  manned 
simulators  will  be  needed  to  improve  real¬ 
ism,  with  the  long  range  goal  being  improve¬ 
ment  of  the  credibility  and  realism  of 
SAFOR.  The  working  group  challenged  the 
T&E  community  at  large  to  develop  and 
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articulate  its  requirements  in  this  area. 

Credibility  and  W&A  of  the  entire  DIS 
network  (all  nodes,  all  models,  all  network 
interactions):  who  does  it  and  what  inter¬ 
agency  and  intra-  and  interservice  agreements 
will  be  required?  Who  is  ultimately  respon¬ 
sible  for  network  management  and  credibility 
of  results,  particularly  in  multiservice  test 
exercises?  Even  though  each  node  may  be 
running  accredited  simulations,  the  interac¬ 
tions  and  dynamics  around  the  network  raise 
unanswered  questions  about  the  VV&A 
process  for  an  interactive  system. 

CONCLUSIONS  AND  RECOMMENDA¬ 
TIONS 

•  The  T&E  community  should  identify 
the  issues  that  should  be  addressed  well 
in  advance  by  DIS  for  the  testing  of  a 


state  that  hardware  simulators  must 


be  DIS-compatible.  With  concurrent 
development  of  DIS  and  the  system  or 
item  being  modeled,  simulations  do  not 
always  represent  the  latest  version  of 
the  system. 

•  For  multiservice  DIS  applications,  the 
lead  agency  "user"  should  review 
W&A  of  all  net  components,  prepare 
the  W&A  section  of  the  TEMP,  de¬ 
scribe  the  limitations  of  the  DIS  appli¬ 
cation,  and  prepare  inter-Service 
MOAs  as  needed.  The  responsibilities 
must  be  clearly  delineated  in  the 
TEMP.  For  single  service  applications 
in  the  future,  pre-accredited  "off  the 
shelf'  modules  should  be  developed 
and  archived  for  classes  of  DIS  appli¬ 
cations.  Pre-accreditation  implies  the 
need  for  an  agency  who  serves  as  the 
repository  of  W&A  and  other  docu¬ 
mentation  for  "approved"  modules. 
The  T&E  community  should  state  their 
data  requirements  now  (e.g.,  formats, 
resolution)  as  most  differ  from  the 
training  community's  needs. 

DIS  should  be  considered  a  test  re¬ 
source  (or  series  of  test  assets)  and  be 
planned  for  and  funded  accordingly. 
DIS  modules  should  be  developed  for 
use  in  COEAs  with  the  intent  of  using 
them  throughout  the  acquisition  cycle 
(through  all  phases  of  T&E). 

DIS/DSI  enhancements  are  required  to 
overcome  latency  problems. 


given  weapon  system.  These  issues  • 
should  be  documented  in  the  Test  and 
Evaluation  Master  Plan  (TEMP). 

•  SAFOR  should  be  enhanced  to  contain 
"pre-blessed"  scenarios,  rules  of  en-  • 
gagement,  tactics,  doctrine,  etc.  The 

T&E  community  should  develop  and 
articulate  its  requirements  for  SAFOR  • 
as  soon  as  possible. 

•  If  DIS  is  to  become  a  viable  test  ad¬ 
junct,  a  new  DoD  Directive  is  needed 
which  will  require  all  system  RFPs  to  • 
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CHAPTER  4 
WORKING  GROUP  3 

OPERATIONS  PLANNING  AND  REHEARSAL 

Commander  Dennis  McBride 

This  working  group  issued  no  final  report. 
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CHAPTERS 
WORKING  GROUP  4 

LOGISTICS,  MOBILIZATION  AND  SUSTAINMENT 

Dr  Lisa  Sokol 


OBJECTIVES 

Our  intent  during  the  working  group  session 
was  to  perform  an  objective  assessment  of 
the  potential  of  Distributed  Interactive  Simu¬ 
lation  (DIS)  for  logistics  applications,  and  to 
assess  the  capability  of  DIS  to  support  cur¬ 
rent  conflicts,  and,  in  particular,  to  assess  the 
capability  of  DIS  to  help  establish  the  rapid, 
flexible  reaction  on  the  part  of  the  logistics 
support  system  required  by  modem  scenar¬ 
ios. 

DISCUSSION  AND  USES 

This  section  presents  a  summary  of  the  dis¬ 
cussions  of  the  working  group  arranged  in  a 
more  logical  sequence  than  that  in  which  they 
occurred. 

A  significant  issue  within  the  working  group 
was  the  definition  of  the  boundaries  of  DIS. 
We  determined  that  two  sets  of  values  are 
indicated  by  the  term  DIS.  The  narrow  set 
relates  to  simulations  such  as  SIMNET  or 
Close  Combat  Tactical  Trainer  (CCTT).  It 
has  the  characteristics  of  human-in-the-loop 
simulations  interacting  with  each  other  to 
perform  common  tasks.  Generally,  the 
human  participation  is  at  the  crew  or  operator 
level,  and  the  human  participants  have  a 
real-time  task  requiring  the  cooperation  of  all 
or  most  of  the  manned  simulators.  The 
second  definition  is  much  broader,  and  wid¬ 
ens  DIS  to  include  any  interactive  simula¬ 
tions  at  any  level.  In  this  case  the  key  char¬ 
acteristic  is  the  presence  of  two  or  more 


independent  simulators,  sharing  common 
data  and  exchanging  results  over  a  network. 
The  group  decided  that  both  definitions 
needed  to  be  considered. 

The  group  considered  both  training  and 
analysis  applications  of  DIS.  It  also  consid¬ 
ered  the  applications  to  Brigade/  Battalion 
echelons,  Division/Corps/Theater  echelons 
and  to  the  mobilization/ sustainment  base. 

Training  Applications 
APPLICATIONS  AT  BRIGADE/  BATTAL¬ 
ION  ECHELONS.  The  operations  of  Brigade 
and  Battalion  S4s  are  properly  the  subject  of 
the  Combat  Arms  proponents.  Our  group 
decided,  however,  that  these  operations 
would  likely  become  orphans  if  we  did  not 
consider  them.  It  was  our  opinion  that  the 
logistics  elements  at  brigade  and  battalion 
levels  could  benefit  from  training  using  DIS 
technology.  In  particular,  we  believe  that  the 
movement  of  such  personnel  into  the  training 
audience  of  exercises  using  SIMNET  like 
simulations  would  be  beneficial  to  both  the 
logistics  and  combat  players.  Rather  than 
using  a  "teleportation"  approach,  logistics 
vehicles  should  be  visible  as  they  move  about 
the  battlefield.  The  forward  logistics  support 
elements  should  also  be  visible  and  active. 
Appropriate  use  of  Semi-  Automated  Forces 
(SAFOR)  would  preclude  excessive  man¬ 
power  or  simulator  requirements.  However, 
the  absence  of  these  elements  hurts  the  pres¬ 
ent  training  conducted  with  DIS. 

In  a  nonlinear  battlefield,  combat  forces 
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should  expect  to  encounter  non-combat 
elements  on  the  battlefield.  It  would  be 
considered  a  tactical  success  to  avoid  direct 
conflict  with  an  opposing  force  and  instead 
destroy  his  logistics  support  capability.  Like¬ 
wise,  our  commanders  must  learn  to  protect 
the  forward  logistics  elements  that  are  other¬ 
wise  vulnerable  to  enemy  action.  The  ab¬ 
sence  of  these  elements  significantly  reduces 
the  realism  of  the  training  presented. 

DIS  (in  the  narrow  sense)  was  also  consid¬ 
ered  as  a  possible  means  to  train  operations 
at  logistics  elements.  For  example,  the 
coordination  of  various  equipment  operators 
at  a  port  facility  might  be  improved  by  train¬ 
ing  in  DIS  simulators  of  their  equipment. 

APPLICATIONS  AT  DIVISION/  CORPS/ 
THEATER  ECHELONS.  The  training  audi¬ 
ences  at  these  echelons  consist  of  managers 
and  decision  makers  who  do  not  directly 
view  the  battlefield,  but  rather  depend  on 
reports  and  analyses  to  support  their  activi¬ 
ties.  As  such,  these  training  audiences  have 
little  to  gain  from  the  narrow  definition  of 
DIS,  but  can  make  use  of  the  wider  defini¬ 
tion.  In  this  manner,  linking  aggregate  level 
simulations  can  provide  significant  advan¬ 
tages  in  training.  As  an  example,  the  group 
discussed  the  Combat  Service  Support  Train¬ 
ing  Simulation  System  (CSSTSS)  and  its 
potential  interfaces.  Interfacing  CSSTSS  to 
the  Corps  Battle  Simulation  (CBS)  is  already 
being  explored.  It  was  generally  agreed  that 
training  at  this  echelon  benefits  from  such 
linkages. 

When  considering  the  linking  of  aggregate 
level  simulations  to  SIMNET-like  versions  of 
DIS,  the  group  saw  little  real  advantage  and 
several  major  disadvantages.  While  the  use 
of  SIMNET  participants  in  a  logistics  exer¬ 


cise  might  seem  to  improve  the  overall  real¬ 
ism,  in  fact,  little  of  the  actions  at  brigade  or 
battalion  would  directly  affect  the  logistics 
training  audience.  It  is  clearly  not  practical 
to  field  a  full  corps  of  manned  simulators. 
Once  significant  numbers  of  SAFOR  are  in 
use,  the  exercise  is  once  again  operating 
more  at  the  aggregated  level.  Data  from  DIS 
exercises  can  be  used  to  calibrate  aggregate 
level  simulations. 

Another  key  limitation  is  that  logistics  prob¬ 
lems  are  generally  many  days  in  duration, 
while  a  tank  battle  at  battalion  level  occupies 
minutes  to  hours.  For  example,  it  may  take 
thirty  to  sixty  days  of  logistics  work  to  pre¬ 
pare  for  a  five  hour  tank  battle.  This  dispar¬ 
ity  in  time  frame  would  make  it  difficult  to 
train  both  tactical  and  logistics  management 
training  audience  elements  at  the  same  time. 
The  tactical  units  can  better  use  their  time  by 
concentrating  on  significant  engagements.  A 
battalion  of  tank  crews  would  gain  little 
value  if  they  were  to  end  up  in  division 
reserve  for  a  week  long  exercise,  doing 
nothing  but  moving  from  assembly  area  to 
assembly  area.  In  order  to  rationalize  this 
disparity  in  simulation  time,  the  logistics 
simulation  must  be  able  to  run  in  a  faster 
than  wall  clock  mode.  Furthermore,  if  the 
simulation  was  fast  enough,  the  users  would 
be  able  to  compare  alternative  strategies 
within  the  allotted  wall  clock  time. 

Analytical  Applications.  The  group  recog¬ 
nized  that  caution  must  be  used  when  using 
training  simulations  to  support  analysis. 
Training  simulations  are  generally  designed 
to  produce  an  environment  in  which  the 
student  can  learn  battlefield  skills.  They  are 
not  intended,  however,  to  predict  the  out¬ 
come  of  any  given  battle.  Only  when  the 
limitations  of  training  simulations  are  fully 
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understood,  can  experiments  be  designed  to 
take  advantage  of  them  in  an  analytical 
mode.  While  this  caveat  has  been  stated,  it 
does  not  preclude  a  careful  approach  to  using 
such  simulators  for  analysis.  The  Louisiana 
Maneuvers  intend  to  use  training  simulations 
for  analytical  purposes.  One  observation  of 
the  results  of  such  analysis  is  that  training 
simulations  may  well  disclose  problems  in  a 
particular  approach.  That  is,  they  may  be 
able  to  demonstrate  that  something  cannot 
work.  However,  only  in  very  carefully 
limited  circumstances  can  they  be  taken  to 
prove  that  a  particular  concept  will  work. 

Examples  of  appropriate  use  of  training 
simulations  for  analysis  are  the  73  Easting 
and  the  demonstrated  WARBREAKER  ap¬ 
proaches.  In  each  case,  the  approach  is  to 
first  recreate  the  historical  situation  in  the 
simulation,  and  then  to  make  careful  excur¬ 
sions  from  the  known  performance.  A  po¬ 
tential  exploitation  of  this  method  of  analysis 
for  logistics  purposes  could  be  the  interface 
of  two  versions  of  CSSTSS.  One  would 
represent  the  European  Theater,  while  the 
other  would  represent  the  Desert  Shield 
logistics  elements.  The  simulators  could  then 
be  used  to  recreate  the  events  involved  in  the 
transfer  of  7th  Corps  from  Europe  to  the 
Saudi  Arabian  Peninsula.  It  is  known  that 
many  problems  occurred,  and  the  use  of  this 
simulation  might  make  possible  the  study  of 
the  problems  to  prevent  their  recurrence. 

CONCLUSIONS 

Many  analytical  models  exist  within  the 
various  logistics  communities,  and  it  was  the 
opinion  of  the  group  that  it  would  be  to  our 
advantage  to  try  to  establish  links  between 
the  best  of  these  models  so  that  an  "end- 
to-end"  simulation  of  the  logistics  process 


could  be  accomplished.  It  was  also  the 
opinion  of  the  group,  that  the  owners  of  such 
models  would  have  reservations  about  their 
use  in  such  a  linkage.  The  group  identified 
a  strong  need  for  interfaces  that  spanned 
service  responsibilities  so  that  both  airlift  and 
sealift  could  be  factored  into  the  simulation 
of  battlefield  logistics. 

The  group  was  unanimous  in  its  belief  that 
the  omission  of  logistic  simulation  compo¬ 
nents  skews  the  outcome  of  the  simulation 
model.  We  must  incorporate  logistics  mod¬ 
els  within  the  context  of  the  battlefield 
model.  Further  research  is  needed  to  deter¬ 
mine  the  level  of  detail  and  fidelity  which 
would  be  appropriate  for  the  logistics  por¬ 
tions  of  the  battlefield  models. 

We  also  recognize  that  different  portions  of 
the  logistics  simulation  model  are  relevant  to 
different  people.  The  portions  of  the  model 
which  run  a  high  aggregation  level,  those 
portions  of  the  model  which  address  strategic 
and  operational  planning,  will  be  most  valu¬ 
able  to  the  logistician.  The  portions  of  the 
model  which  run  a  low  level  of  aggregation, 
the  levels  which  focus  on  details,  will  be 
most  relevant  to  the  tactical  forces. 

We  believe  that  the  visualization  portions  of 
the  model  are  critical  to  the  success  of  the 
model.  Furthermore,  the  appropriate  visual¬ 
ization  tools  will  vary  with  the  user. 

Finally,  we  recognize  the  large  disparity  that 
exists  between  the  time  frame  of  the  typical 
battlefield  model  and  the  time  required  to 
plan  the  logistics  support  and  resupply  for  a 
battle.  In  order  to  manage  this  disparity  of 
temporal  interests,  the  logistics  portion  of  the 
battlefield  must  be  run  in  faster  than  real 
time.  It  would  also  be  useful  to  be  able  to 
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run  several  iterations  or  projections  of  alter¬ 
natives  within  the  appropriate  time  steps. 

RECOMMENDATIONS 

The  following  is  a  list  of  recommendations 
that  were  generated  as  a  result  of  the  work¬ 
shop: 

•  The  logistics  community  must  begin  to 
develop  an  appropriate  module  for  the 
current  set  of  virtual  simulations. 

•  The  time  frame  for  battlefield  simula¬ 
tions  must  be  extended  to  introduce 


interactions  with  logistics  capabilities. 

•  The  time  frame  of  logistics  simulations 
relative  to  that  of  battlefield  simula¬ 
tions  makes  it  imperative  to  introduce 
faster  than  real  time  simulation. 

•  Logistics  simulation  fidelity  and  speed 
can  be  enhanced  by  research  in  aggre¬ 
gation  and  disaggregation  techniques. 

•  The  logistics  community  should  begin 
to  try  to  establish  links  between  the 
best  of  the  logistics  models  to  create  an 
"end-to-end"  simulation  of  the  logistics 
process. 
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CHAPTER  6 
WORKING  GROUP  5 
FORCE  DEVELOPMENT 
Dr  Darrell  Collier 

This  working  group  issued  no  final  report. 
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CHAPTER  7 
WORKING  GROUP  6 
COMBAT  DEVELOPMENT 
Michael  Bauman 


Although  Working  Group  6  was  assigned  the 
topic  "Combat  Developments,"  the  group 
members  were  not  inhibited  about  encroach¬ 
ing  into  the  topic  areas  assigned  to  other 
working  groups.  This  was  rightfully  so  since 
Combat  Developments  (CD)  embraces  the 
development  of  concepts  and  doctrine,  design 
of  forces,  definition  of  system  requirements, 
test  and  evaluation;  while  at  the  same  time 
cutting  across  the  domains  of  combat  arms, 
logistics  and  military  personnel. 

Working  Group  6  (WG6)  was  comprised 
largely  of  hard-core  practitioners,  which 
helped  keep  the  group  firmly  anchored  to  the 
reality  of  military  operations  research  and 
focused  "where  the  rubber  meets  the  road." 
The  group  also  benefited  by  having  several 
members  who  have  played  instrumental 
supporting  roles  in  the  development  of  DIS. 
Although  enthusiastic  about  the  tremendous 
potential  DIS  offers  to  Combat  Develop¬ 
ments,  the  group  was  not  blinded  by  its 
visual  allure.  To  a  disciplined  OR  practitio¬ 
ner,  seeing  is  not  necessarily  believing, 
whether  it  is  DIS  or  another  venue.  Guarded 
optimists  best  describes  persons  in  WG6. 

Following  opening  introductions,  all  of  WG6 
met  together  to  explore  the  potential  uses  and 
limitations  of  applying  DIS  to  CD.  These 
ideas  were  then  grouped  into  major  headings 
and  discussed  in  more  detail  within  four 
smaller  groups  to  stimulate  greater  discussion 
by  everyone.  At  the  close,  all  of  WG6  met 
together  again  to  share  individual  positions 
and  findings,  and  to  contribute  to  the  final 


report. 

DIS  POTENTIAL 

Three  key  potentials  dominated  our  discus¬ 
sion  of  DIS  applied  to  CD.  The  first  and 
most  often  cited  potential  was  the  opportunity 
to  embed  actual  warfighters  and  operators 
into  the  simulated  environment  as  an  integral 
part  of  the  weapon  system  design,  particu¬ 
larly  during  the  system's  early  development, 
but  also  throughout  the  complete  develop¬ 
ment  process  leading  to  fielding.  This  recog¬ 
nizes  the  advantages  held  by  virtual  simula¬ 
tions  (simulators)  over  other  simulation 
components  of  DIS,  e.g.,  the  introduction  of 
human  operators,  hands-on,  in-control, 
reacting  to  the  environment,  making  real¬ 
time  decisions  and  taking  actions  which 
influence  the  simulation  outcomes.  WG6 
believed  this  potential  will  greatly  aid  combat 
developers  in  early  system  design  such  as 
man-machine  interfacing  and  early  evaluation 
of  new  operational  and  tactical  concepts. 

The  second  key  potential  also  pertains  to  the 
human  dimension  of  DIS.  The  ability  to 
monitor  and  record  events  in  virtual  simula¬ 
tions  (simulators),  particularly  human  actions 
and  reactions  within  the  simulated  environ¬ 
ment,  adds  a  new  powerful  dimension  to  the 
evaluation  of  human  performance  and  after- 
action  review  of  simulated  exercises  or  trials, 
compared  to  either  constructive  simulations 
(without  a  human-in-control)  or  testing  and 
live  simulations  (where  instrumentation  may 
be  limited  in  its  ability  to  monitor  and  record 
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crew  member  or  operator  actions).  The 
virtual  simulations  are  more  controllable  than 
a  field  test  or  exercise,  yet  still  permit  the 
human  crewmember  or  operator  to  interact 
with  the  environment,  something  that  is 
missing  in  constructive  simulations. 

The  third  key  potential  is  the  facilitation  of 
joint-ness  in  combat  developments.  Until 
recently,  below  the  JCS  level,  the  joint-ness 
depicted  in  each  of  our  service  simulations 
has  been  largely  the  result  of  the  individual 
service  (who  owned  the  simulation),  rather 
than  the  joining  of  each  service's  perspective. 
The  promise  of  DIS  to  seamlessly  integrate 
heretofore  disparate,  stand-alone  simulations 
across  service  boundaries  may  create  new 
opportunities  for  interservice  CD.  WG6 
recognized  that  "turf"  obstacles  must  still  be 
overcome  to  achieve  Joint  CD  in  a  practical 
way;  that  is,  just  because  it  can  be  done 
doesn't  guarantee  it  will  ever  happen.  Nev¬ 
ertheless,  the  DIS  movement  will  promote 
greater  dialogue  and  exchange  among  service 
and  industry  modelers  as  they  prepare  their 
particular  simulations  for  linkage  within  DIS. 
In  mm,  this  should  make  it  easier  to  create  a 
simulation  environment  comprised  of  the  best 
available  representations  of  military  sea, 
land,  air  and  space  functions.  This  will 
facilitate  joint-ness  in  CD,  whether  it  is  joint 
CD  conducted  by  or  on  behalf  of  JCS,  OSD 
or  multiservices,  or  simply  CD  conducted  by 
or  for  a  single  Service. 

CONCERNS:  CREDIBILITY  AND  MAN¬ 
AGEABILITY 

WG6  addressed  a  multitude  of  real  and 
potential  problems  related  to  using  DIS  for 
CD.  For  the  most  part,  the  problems  were 
linked  to  two  practical  concerns:  credibility 
and  manageability.  Under  the  first,  WG6 


repeatedly  surfaced  issues  about  Verification, 
Validation,  and  Accreditation  (W&A),  and 
Configuration  Management.  W&A  was  the 
#1  vote  getter  as  a  problem  area.  Under  the 
second,  WG6  identified  a  variety  of  issues 
pertaining  to  priority  of  usage,  accessibility, 
and  funding.  In  addition,  the  group  ex¬ 
pressed  dismay  at  the  technical  challenges 
confronting  the  realization  of  DIS  to  its 
fullest,  most  often  mentioning  the  linking  of 
heterogenous  simulations.  The  group  also 
worried  that  high-visibility  weaponeering 
simulations  will  dominate  as  they  have  in  the 
past  the  evolution  of  DIS  at  the  expense  of 
achieving  a  balanced,  integrated  combined 
arms  simulation  capability. 

Credibility  issues.  Credibility  is  absolutely 
vital  to  the  use  of  DIS  in  CD.  W&A  is  the 
prerequisite.  Ultimately,  the  business  of  CD 
is  about  making  important  choices  (What 
doctrine  and  tactics?  How  to  structure? 
Where  to  invest  in  modernization?).  The 
decision  makers  who  make  the  choices  must 
have  confidence  in  the  evidence  presented  to 
them.  The  OR  practitioners  must  instill  that 
confidence.  The  underlying  modeling  must 
represent  the  full  functionality  necessary  for 
the  particular  CD  application.  Combat 
cause-and-effect  relationships  must  be  prop¬ 
erly  accounted  for.  The  model  data  must  be 
accurate  and  up-to-date.  These  requirements 
continue  across  the  spectrum  of  verification, 
validation,  and  accreditation  (W&A)  issues. 

WG6  asserted  that  V&V  is  still  the  responsi¬ 
bility  of  the  individual  Service  modelers 
(those  who  are  proponents  of  the  simulation), 
even  under  DIS;  and  that  Accreditation  still 
belongs  to  the  customer  of  the  simulation- 
based  work.  However,  being  part  of  DIS 
carries  extra  W&A  burdens.  For  example, 
the  unique  interface  software  between  a 
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particular  simulation  and  the  DIS  network 
(e.g.,  Cell  Adaptor  Unit  or  CAU)  must  also 
be  subjected  to  W&A.  When  two  or  more 
simulations  linked  on  the  network  are  model¬ 
ing  the  same  functions  (and  most  likely  they 
will),  precedents  must  be  established  for 
dominance  of  events  and  outcomes.  The 
W&A  of  Semi-Automated  Forces  (SAFOR) 
or  Computer-Generated  Forces  (CGF)  was 
particularly  worrisome  to  WG6  to  include  the 
representation  itself  and  its  consistency  with 
virtual  (manned)  simulations.  There  was 
partial  agreement  that  a  single  service  agency 
(or  a  union  of  a  few)  should  be  made  respon¬ 
sible  for  SAFOR/CGF. 

The  group  saw  great  merit  in  establishing  and 
publishing  DIS-wide  standards  for  particular 
sub-models  and  algorithms  embedded  within 
individual  simulations,  as  a  function  of  gran¬ 
ularity  or  resolution  (e.g.,  6-degree  of  free¬ 
dom  (DOF)  engineering,  item-system,  sortie, 
battalion).  The  standards  could  address  how 
to  model  target  acquisition  from  air  to 
ground,  how  to  adjudicate  direct  fire  engage¬ 
ments,  how  to  model  mobility  of  ground 
vehicles,  etc.  This  approach  would  support 
V&V  in  a  building  block  fashion  from  the 
bottom-up,  help  to  overcome  the  shortcom¬ 
ing  of  stovepiped  V&V  of  individual  simula¬ 
tions  by  separate  agencies,  and  promote 
greater  consistency  among  the  simulation 
linked  on  the  network. 

WG6  believed  industry  could  play  a  greater 
role  in  W&A,  and  would  do  so  enthusiasti¬ 
cally  when  offered  the  opportunity.  Industry 
has  a  vested  interest  in  the  modeling  of  the 
functional  areas,  e.g. ,  air-to-air  engagement, 
logistics  resupply,  where  they  are  investing 
or  competing.  Why  not  let  industry  firms 
review  and  provide  an  assessment  of  the 
simulation  within  their  particular  areas  of 


expertise  to  support  the  overall  V&V  effort? 

Finally,  in  support  of  W&A,  WG6  con¬ 
cluded:  Document  ...  Document  ...  Docu¬ 
ment!  Always  important,  simulation  docu¬ 
mentation  is  even  more  important  under  DIS 
because  the  W&A  process  is  no  longer 
confined  to  the  simulation  owner  and  a  few 
select  customers.  With  DIS,  there  are  many 
potential  "co-owners"  and  customers  of  a 
particular  simulation.  Each  DIS  player  must 
have  sufficient  information  via  documenta¬ 
tion  in  order  to  intelligently  select  those 
simulations  with  which  to  link,  understand 
what  they  have  linked  into,  and  have  confi¬ 
dence  about  the  results  they  got  from  it. 

Management  Issues.  WG6  surmised  a  num¬ 
ber  of  ways  that  DIS  could  be  mismanaged, 
mostly  based  on  first-hand  experience  with 
today's  simulations,  and  extrapolating  into  a 
DIS  environment  where  "demand-exceeds- 
supply"  and  budgets  are  shrinking.  There 
are  already  indicators  that  the  Army's  grow¬ 
ing  capability  in  virtual  simulations  is  not 
keeping  pace  with  the  demand  for  access  to 
it. 

WG6  concluded  that  DoD  should  fund  the 
continued  evolution  and  operations  and 
maintenance  (O&M)  of  DIS  which  are  "in 
common"  to  DoD,  JCS,  CINCs,  and  the 
services;  and  that  customers  with  unique 
needs  or  applications  should  pay  their  own 
way.  The  funding  issue  was  particularly 
worrisome  to  WG6  because  they  exceed  the 
expertise  of  most  organic  resources  and  will 
require  contracted  augmentation.  Also,  a 
key  potential  of  DIS  is  in  early  concept 
development,  before  a  funded  program  ex¬ 
ists.  "Where  will  the  funds  come  from?" 
was  the  million-dollar  question  with  no 
satisfactory  answer. 
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The  deliberate  master  planning  and  schedul¬ 
ing  of  utilization  of  DIS-related  resources 
(network,  nodal  simulations,  players  over  a 
planning  horizon  of  several  years)  were  cited 
as  critical.  WG6  believed  the  network 
should  be  centrally  managed  while  nodal 
assets  should  be  de-centrally  owned  and 
controlled.  There  must  be  a  disciplined 
planning  process  by  which  to  access  the 
resources,  and  to  de-conflict  competing 
demands  for  their  use.  In  the  event  that 
schedules  are  disrupted  and  impact  costs  are 
incurred,  the  out-of-cycle  users  (queue  jump¬ 
ers)  should  reimburse  the  expenses  of  those 
affected.  The  DIS  network  needs  a  dedicated 
scheduling  mechanism  similar  to  the  CINCs' 
exercise  schedule  or  the  Army's  Five-Year 
Test  Program  (FYTP). 

WG6  believed  that  DIS  should  be  available 
to  anyone  with  a  need  to  use  it  and  a  means 
to  pay  for  it,  to  include  private  industry, 
FFRDCs,  and  non-governmental  agencies 
such  as  universities  and  research  centers. 
The  general  consensus  was  DIS  will  be 
enriched  by  the  diversity  of  users.  It  should 


not  be  free;  a  charge-back  fee  to  non-govern¬ 
ment  users  would  offset  the  cost  of  DIS 
O&M  and  also  curtail  frivolous  demands  for 
access.  Prudent  management  of  network 
access  and  encryption  may  be  sufficient  to 
protect  a  firm's  proprietary  information 
placed  at  risk  while  using  DIS. 

CONCLUSIONS 

DIS  offers  tremendous  potential  for  combat 
developments,  but  not  without  a  price  to  pay 
and  many  challenges  to  overcome,  both 
technical  and  managerial.  Proof-of-principle 
demonstrations  and  video  shows  are  inade¬ 
quate  to  build  the  experience  and  confidence 
necessary  to  exploit  DIS  for  CD.  The  key  is 
relentless  W&A.  To  realize  its  full  poten¬ 
tial,  DIS  must  be  applied  to  real  and  tough 
problems  where  it  will  be  subjected  to  critical 
OR  practitioners,  demanding  military  war¬ 
fighters  and  users.  It  must  be  challenged  by 
minimal-risk  decision  makers  seeking  to 
reduce  risk  for  billion-dollar  decisions.  We 
all  have  to  roll  up  our  sleeves  and  make  DIS 
work  for  us. 
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CHAPTER  8 
WORKING  GROUP  7 
TRAINING  AND  READINESS 

Kenneth  Lavoie 


FOCUS 

Investigate  the  use  of  DIS  for  cost  effective 
training  and  readiness. 

SUGGESTED  TOPICS 

Training  multiservice  teams;  adequate  levels 
of  operational  realism;  potentials  for  and 
safeguards  against  negative  training;  proper 
applications  of  semi-automated  forces 
(SAFOR);  additional  features  desired  to 
optimize  training  impact;  training  density 
issues;  command  and  control  considerations; 
potential  for  interfacing  with  mission  task 
trainers  and  other  training  devices;  utility  for 
reserve  components;  training  and  orientation 
of  military  personnel  to  DIS  applications; 
electronic  warfare  and  intelligence  operation; 
exploiting  prototype  simulators  as  training 
devices.  Requirements  such  as  terrain  reso¬ 
lution,  and  data  collection  and  reduction 
issues  in  support  of  performance  and  effec¬ 
tiveness  measures. 

QUESTIONS  EXAMINED 

After  initial  discussions  to  determine  general 
areas  of  interest  among  the  participants,  the 
suggested  topic  areas  were  restructured  under 
four  major  issues:  training  effectiveness, 
simulator  realism,  joint  and  multiservice 
applications,  and  semi-automated  forces 
(SAFOR).  This  grouping  did  not  eliminate 
the  remaining  topic  areas,  but  clearly  focused 
the  discussions  on  major  areas  of  group 
interest.  As  a  matter  of  working  vocabulary, 


the  group  limited  its  deliberations  on  DIS  to 
virtual  simulations.  The  focus  upon  manned 
simulators  engaged  on  wholly  synthetic 
computer  generated  battle  environments 
seemed  the  principal  application  of  DIS. 
There  are  clearly  other  more  inclusive  inter¬ 
pretations  of  DIS  which  encompass  both 
constructive  or  computer  combat  models,  and 
live  or  field  training  exercises.  It  is  impor¬ 
tant  in  reading  this  report  to  recognize  the 
limits  placed  by  the  group  on  the  range  of 
application  of  the  DIS  terminology. 

SPECIAL  WORKING  GROUP  BRIEF¬ 
ING 

After  determining  the  four  major  topic  areas 
that  the  group  would  focus  on,  two  briefings 
were  provided  by  group  members.  COL 
Overstreet  provided  a  briefing  on  the  Army's 
Close  Combat  Tactical  Trainer  project.  The 
briefing  covered  the  objectives,  development 
schedule,  and  provided  insights  on  the  near 
term  technology  to  be  implemented.  The 
second  briefing  by  Dr.  Herb  Bell  of  the  Air 
Force  Armstrong  Laboratory  addressed 
ongoing  efforts  to  apply  DIS  technology  to 
Air  Force  training  needs. 

On  the  topic  of  SIMNET  or  DIS  protocols, 
the  work  by  Armstrong  Labs  identified  the 
need  for  additional  data  elements  to  be  incor¬ 
porated  in  each  packet  describing  a  fast 
moving  airborne  vehicle.  These  additional 
elements  were  necessitated  by  the  increased 
speed  and  three  dimensional  travel  of  the 
vehicles. 
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Work  done  at  Armstrong  Labs  with  front  line 
pilots  provided  significant  insights  into  the 
applicability  of  DIS  to  aircrew  training. 

It  was  found  that  for  air-to-air  engagements 
occurring  beyond  visual  range,  or  at  ex¬ 
tended  visual  ranges,  the  simulators  were 
capable  of  providing  a  positive  contribution 
to  aircrew  training.  However,  once  the 
combat  range  collapsed  to  close-in  encoun¬ 
ters,  the  importance  of  near  instantaneous 
position  updates,  coupled  with  the  need  for 
near  perfect  visual  representation  to  support 
visual  cueing,  exceeded  the  capabilities  of  the 
simulators. 

Similarly,  it  was  found  that  the  requirements 
for  graphical  representation  of  terrain  and 
target  features  precludes  the  use  of  current 
systems  for  air-to-surface  weapons  delivery 
training. 

It  was  also  confirmed  during  the  experiments 
that  there  is  a  significant  risk  of  negative 
training  when  simulators  are  used  for  tasks  in 
which  their  fidelity  is  inadequate.  Pilots 
attempt  to  use  the  information  provided  by 
the  simulator  and  modify  real  world  proce¬ 
dures  to  compensate  for  simulator  short  falls. 
This  is  a  fundamental  danger  zone  for  nega¬ 
tive  training  to  be  watched  for  with  great 
diligence. 

It  would  appear,  therefore,  those  SAFOR 
that  can  credibly  replicate  the  performance, 
impact,  and  C2  relationships  of  tactical  air- 
power  would  be  prime  research  areas  for  the 
support  of  joint  operations. 

GENERAL  APPROACH 

The  evolution  of  DIS  capability  from  its 
genesis  in  SIMNET  has  been  significantly 


influenced  by  technology.  To  the  point  that 
it  is  probably  safe  to  say  that  DIS  is  the 
result  of  a  "technology  push"  effort.  For  this 
reason  the  working  group  adopted  the  con¬ 
trary  view  of  "requirements  pull"  in  discus¬ 
sions  of  the  applications  and  purposes  of 
DIS.  In  fact,  some  time  was  spent  discussing 
the  DIS  vision  that  has  appeared  in  the  litera¬ 
ture  for  a  seamless  exercise  spanning  the 
entire  command  and  control  structure  from 
theater  commander  to  the  individual  weapon 
system  operator  —  the  "god  to  squad"  view. 
The  discussions  within  the  working  group  did 
not  support  a  training  requirement  served  by 
this  capability.  Because  of  the  significant 
differences  in  planning  horizons,  tempo  of 
operations,  information  needs,  and  funda¬ 
mental  training  focus  at  the  various  levels 
within  the  command  and  control  structure, 
we  saw  no  direct  training  benefit  from  such 
an  exercise  structure.  In  fact,  it  was  more 
likely  that  there  could  be  disadvantages  to 
such  an  expansive  training  exercise.  The 
inability  of  various  echelons  to  pause,  reex¬ 
amine,  or  accelerate  the  exercise  as  necessary 
for  the  accomplishment  of  their  specific 
training  objectives  because  of  the  linkage  to 
other  echelons  of  the  C2  structure  was  seen  as 
a  disadvantage. 

FINDINGS  -  TRAINING  EFFECTIVE¬ 
NESS  AND  SIMULATOR  REALISM 

The  discussions  on  Training  Effectiveness 
and  Simulator  Realism  followed  fairly  classi¬ 
cal  lines  emphasizing  the  need  to  clearly 
articulate  and  understand  the  training  objec¬ 
tives,  as  well  as  carefully  defining  the  target 
training  audience.  The  training  equivalent  of 
defining  the  problem  seems  even  more  im¬ 
portant  in  a  discussion  of  DIS  where  there  is 
clearly  the  danger  of  becoming  infatuated 
with  the  technology  and  ending  up  applying 
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Table  2-1.  Training  Objectives 


-  Exercise  Decision  Making 

-  Safety/Constraint  Relief 

-  Develop  Staff  Procedural  Proficiency 

-  Large  Group  Participation 

-  Evaluate  OPLANs 

-  Employ  EW  Systems 

-  Examine  Strategy  Options 

-  Ease  Maneuver  Limits 

it  inappropriately.  With  training  objectives 
clearly  defined  for  a  specific  training  audi¬ 
ence,  the  final  stage  of  problem  definition  is 
to  evaluate  the  relative  costs  and  benefits 
associated  with  DIS  versus  other  training 
opportunities. 

FINDINGS  -  JOINT  AND  MULTI¬ 
SERVICE  APPLICATIONS 

As  stated  previously,  a  requirements  based 
viewpoint  was  adopted  to  determine  the  most 
effective  applications  of  DIS  technology  in 
the  training  area.  Broad  training  objectives 
were  developed  from  both  the  theater  CINC 
perspective  and  from  a  small  unit  or  weapon 
system  operator  perspective.  The  objectives 
are  provided  in  Table  2-1. 

The  most  striking  aspect  of  this  categoriza¬ 
tion  of  objectives  is  the  lack  of  common 
training  objectives  for  the  two  groups. 
Thus, our  discussions  lead  to  the  general 
consensus  that  large  combined  arms  and 


multiservice  training  exercises  were  most 
efficiently  and  effectively  accomplished  using 
constructive  simulation.  Smaller  training 
audiences  composed  of  teams,  new  systems, 
or  joint  task  forces  were  the  training  areas 
best  suited  to  capitalize  upon  the  capabilities 
of  DIS.  Individual  weapon/ vehicle  skills  or 
crew  task  proficiency  training  constituted  the 
third  group  of  training  events.  These  were 
considered  to  be  best  accomplished  by  field 
exercise  or  part  task  trainers.  This  categori¬ 
zation  is  captured  in  Table  2-2. 

FINDINGS 

A  considerable  amount  of  time  and  effort 
was  devoted  to  defining  the  requirements  for 
semi-automated  forces  (SAFOR).  The  abil¬ 
ity  of  the  SAFOR  to  demonstrate  the  re¬ 
quired  degree  of  realism  in  their  actions  and 
reactions  are  critical  to  DIS  being  a  viable 
tool  for  effective  training.  Figure  2-1  identi¬ 
fies  the  fundamental  components  of  a 
SAFOR  system.  Specific  functional  charac- 


Table  2-2.  DIS  Training  and  Readiness 


Type/Level  of  training 

Appropriate  Technology 

Comments 

Combined  Arms  Multiservice 

Constructive  Simulation 

“Joint”  does  not  equal  “large” 

Teams 

DIS 

Tasks  requiring  external  sim- 

Systems 

ulation  are  good  candidates 
for  DIS 

Individual  Skills 

Weapons/vehicle  trainer 

DIS  not  cost  effective  for  op¬ 

Crew  Tasks 

Embedded  trainers 

Live  field  training 

eration  skill 
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Figure  2-1.  Data  Collection 


teristics  of  SAFOR  systems  are  as  follows: 

•  Being  indistinguishable  from  manned 
systems  is  critical  to  the  accomplish¬ 
ment  of  training  objectives. 

•  A  data  collection  capability  is  needed 
to  capture  simulated  communications 
among  SAFOR  as  well  as  between 
SAFOR  and  live  units. 

•  The  decision  processes  affecting 
SAFOR  actions  must  be  captured  and 
be  available  for  debriefing. 

•  Human  factors  such  as  morale,  fatigue, 
and  indecision  must  be  exhibited  by 
SAFOR. 

•  Tactics,  perceived  objectives,  rules  of 
engagement,  and  decision  processes 
must  be  adaptable  to  the  specific  en¬ 
emy  consistent  with  the  training  sce¬ 
nario. 

•  The  span  of  control  of  SAFOR  opera¬ 
tors  must  be  increased  significantly  so 
that  units  can  train  in  large  scale  opera¬ 
tions  without  excessive  controller 
costs. 

•  Identification  issues  of  friend,  foe  or 
neutral  systems  must  not  be  automati¬ 
cally  resolved  either  among  or  between 
SAFOR. 

•  Terrain  avoidance  and  dynamic  terrain 
representation  are  improvements  neces¬ 
sary  to  realize  the  full  training  potential 
of  DIS. 

•  The  need  to  validate  the  performance 
of  the  SAFOR  should  be  an  integral 


part  of  the  basic  design  to  ensure  that 
the  necessary  data  are  explicitly  repre¬ 
sented  and  captured  for  review. 

AREAS  FOR  FURTHER  RESEARCH 

During  the  discussions  about  the  four  major 
topics  above,  a  number  of  issues  were  found 
to  require  further  research  to  determine  if 
they  constitute  real  issues  or  to  advance  the 
current  state  of  their  representation  within 
DIS. 

Topics  Requiring  Basic  Research.  (1) 
Measurement  of  Training  Value:  While  the 
participants  were  convinced  that  DIS  offers 
great  training  potential,  the  practicality  of 
quantifying  the  improvement  in  training  was 
an  issue.  It  was  central  to  the  overall  ap¬ 
proach  to  the  training  and  readiness  issue  that 
cost  benefit  comparisons  be  done  between 
DIS  supported  training  events  and  other 
training  options.  It,  therefore,  seems  prudent 
to  commission  a  study  to  identify  those 
training  events  for  which  DIS  provides  the 
most  cost  efficient  training  approach. 

(2)  Real  vs.  Generic  Data  and  Positive  Vs. 
Negative  Training:  In  many  instances  the 
training  community  is  either  limited  to  using 
unclassified  data,  or  actually  prefers  using 
unclassified  data.  Discussions  centered  on 
the  training  implications  of  using  less  accu¬ 
rate  data  (tacit  assumption  being  that  classi¬ 
fied  information  is  more  accurate)  for  train¬ 
ing  purposes  and  whether  this  would  have  a 
practical,  adverse  effect.  The  fundamental 
concern  was  for  those  situations  where  the 
tactical  implication  of  a  weapon  system 
would  be  altered  by  the  operator  based  upon 
erroneous  data  in  the  system  representation. 

(3)  Train  the  Trainer:  The  extent  of  the 
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training  program  that  will  be  required  to  train 
the  personnel  who  will  operate  the  DIS  was 
another  area  of  discussion  and  concern. 

Topics  Requiring  Further  Research.  (1) 

Bandwidth:  Despite  assurances  to  the  con¬ 
trary,  it  appears  prudent  to  consider  the 
impact  upon  bandwidth  of  increased  data 
elements  necessary  to  accommodate  fixed 
wing  aircraft,  and  significantly  increased 
systems  represented  by  either  SAFOR  or 
manned  simulators. 

(2)  Multi-Level  Security:  The  ability  to 
more  precisely  manage  access  and  use  of 
accurate  classified  data  on  the  network  would 
be  a  considerable  assist  in  dealing  with  the 
use  of  inaccurate  unclassified  data. 

(3)  EW  Representation:  The  current 

representation  of  Electronic  Warfare  (EW) 
needs  to  be  enhanced  to  ensure  the  most 
effective  training.  Training  in  a  representa¬ 
tive  EW  environment  can  be  of  considerable 
benefit  in  minimizing  the  disruptive  impact 
of  real  world  exposure  to  EW  and  can  greatly 
enhance  the  awareness  of  system  operators  to 
existing  work  around  procedures  to  minimize 
the  impact  of  EW  on  operations. 


(4)  Dynamic  Terrain:  The  ability  of  the 
terrain  to  reflect  the  impact  of  operations  is 
important  to  the  continued  improvement  of 
training  fidelity. 

(5)  Debriefing  System:  A  first  class  debrief¬ 
ing  capability  is  key  to  maximizing  the  train¬ 
ing  potential  of  DIS.  The  requirements  for 
such  a  system  should  be  integrated  into  the 
design  of  the  overall  DIS,  as  was  suggested 
in  the  design  of  the  SAFOR. 

CONCLUSIONS  AND  RECOMMENDA¬ 
TIONS 

•  DIS  shows  considerable  promise  in  the 
training  arena  particularly  for  small 
unit  to  joint  task  force  operations. 

•  Joint  training  objectives  should  be 
defined  which  have  been  specifically 
identified  as  being  most  efficiently 
accomplished  using  DIS. 

•  There  will  remain  training  events  that 
are  best  suited  to  live,  DIS,  or  con¬ 
structive  model  supported  exercises. 
The  specific  areas  of  application  of  the 
three  exercise  types  should  be  studied 
and  guidelines  provided  to  the  training 
community. 
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CHAPTER  9 
WORKING  GROUP  8 

REQUIREMENTS  DEVELOPMENT  AND  DEFINITION 

Dr  James  Metzger 


PURPOSE 

As  stated  in  the  Terms  of  Reference,  Work¬ 
ing  Group  8  was  responsible  for  investigating 
the  use  of  DIS  as  the  basis  for  experiments  to 
determine  whether  a  conceptual  materiel 
system  can  satisfy  an  identified  need  and 
what  the  performance  characteristics  of  such 
a  materiel  system  should  be.  The  actual 
discussions  of  the  Working  Group  focused  on 
the  topic  of  the  use  of  DIS  to  support  the 
overall  Acquisition  Management  process; 
that  topic  is,  therefore,  the  subject  of  this 
chapter. 

BACKGROUND 

Features  of  DIS  that  are  useful  for  supporting 
the  Acquisition  Management  process  include: 

•  DIS  allows  warriors  to  be  active 
participants,  thereby  enhancing  the 
acceptability  of  results. 

•  Due  to  its  interactive  nature  -- 

DIS  allows  for  reactive  activities, 
for  both  friendly  and  enemy 
forces,  thereby  again  enhancing 
acceptability. 

DIS  provides  for  visualization  and 
play  back,  thereby  facilitating  as¬ 
sessments  of  cause  and  effect. 

-  DIS  can  be  used  as  a  calibration 
tool  for  the  representation  of  com¬ 
mand  and  control  in  constmctive 
M&S. 

•  DIS  permits  the  linking  of  diverse 
simulators  and  simulations. 


•  DIS  provides  a  less  costly  alternative  to 
live  exercises. 

ESSENTIAL  TECHNICAL  AREAS 

It  is  essential  that  DIS  be  capable  of  provid¬ 
ing  assistance  in  the  following  technical 

areas: 

•  Determining  a  mission  need.  This 
includes  assessment  of  the  contribu¬ 
tions  of  doctrine,  organization,  train¬ 
ing,  leader  development,  and  user,  as 
well  as  the  contribution  of  a  materiel 
system. 

•  Defining,  refining,  and  assessing  the 
operational  requirements  for  a  materiel 
system. 

•  Developing,  refining,  and  assessing 

performance  characteristics  of  a  ma¬ 
teriel  system.  Note  that  this  require¬ 
ment  demands  significantly  more  fidel¬ 
ity  than  does  dealing  with  operational 
requirements. 

POTENTIAL  TECHNICAL  AREAS 

DIS  may  be  helpful  in  the  following  areas: 

•  Determining  training  characteristics  for 
a  materiel  system. 

•  Determining  supportability  require¬ 

ments  for  a  materiel  system. 

•  Determining  requirements  for  inter¬ 

operability  of  a  materiel  system  with 
other  systems. 
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FUNDAMENTAL  CAPABILITIES 


In  order  to  be  able  to  provide  assistance  in 

the  essential  technical  areas  identified  above, 

DIS  must  have: 

•  Validated  representations  of  battlefield 
activities  and  materiel  systems. 

•  Extensive  pre-  and  post-processors. 

•  A  realistic  environment.  It  must  be 
robust,  portable,  common  to  all  partici¬ 
pants,  and  capable  of  representing  joint 
activities. 

•  Extensibility,  both  in  terms  of  types 
and  numbers  of  materiel  systems  repre¬ 
sented  and  in  terms  of  the  level  of 
detail  for  individual  systems. 

•  Well-defined  interfaces,  standards,  and 
protocols. 

•  "Good"  fidelity  at  varying  levels  of 
resolution.  Here  "good"  is  determined 
by  the  particular  application;  e.g.,  the 
fidelity  must  be  greater  for  dealing 
with  performance  characteristics  than 
for  dealing  with  operational  require¬ 
ments.  In  conjunction  with  extensibil¬ 
ity  in  level  of  detail,  this  argues  that 
simulators  and  protocols  should  be 
designed  in  such  a  way  that  resolution 
can  be  varied  depending  on  the  appli¬ 
cation. 

•  Modular,  adaptive  simulators.  This 
means  that  individual  simulators  can  be 
easily  reconfigured  to  represent  new  or 
revised  materiel  system  capabilities, 
and  possibly  could  mean  generic  simu¬ 
lators  that  can  be  configured  to  repre¬ 
sent  a  variety  of  systems  and  associated 
capabilities. 

•  A  full-time  support  staff  for  DIS  cen¬ 
ters. 

•  High  reliability  for  all  components  of 
DIS,  including  individual  simulators, 


communications,  computers,  and  soft¬ 
ware. 

CAVEATS 

The  positive  potentials  of  DIS  must  be  tem¬ 
pered  with  realism: 

•  Development  of  DIS  with  the  "good" 
fidelity  necessary  for  support  to 
Acquisition  Management  may  require 
significant  up-front  investment. 

•  The  use  of  DIS  can  place  significant 
demands  for  personnel  resources;  i.e., 
participants,  as  well  as  support  staff 
such  as  controllers  and  data  base  ex¬ 
perts. 

•  DIS  is  but  one  tool  for  supporting 
Acquisition  Management.  It  cannot 
answer  all  questions. 

•  DIS  lacks  repeatability,  thereby  com¬ 
plicating  experimental  design  and  po¬ 
tentially  masking  cause  and  effect 
relationships. 

CONCLUSION 

DIS  has  potential  value  in  supporting  the 

Acquisition  Management  process,  but  may 

require  significant  investment  to  be  useful. 

RECOMMENDATIONS 

•  Perform  research  to  identify  the  level 
of  detail  needed  to  provide  "good" 
fidelity  for  the  use  of  DIS  in  support  of 
Acquisition  Management. 

•  Perform  a  cost-benefit  analysis  to  ad¬ 
dress  the  costs  and  benefits  of  DIS  in 
support  of  Acquisition  Management. 

•  Invest  in  the  following  fundamental 
capabilities  now: 

Validated  representations. 


-  Pre-  and  post-processors. 

-  A  realistic  environment. 

-  Extensibility. 

•  Support  the  proposed  proof-of-princi- 
ple  project  initiated  by  the  Office  of 


the  Assistant  Secretary  of  Defense 
(Program,  Analysis  and  Evaluation) 
and  the  Defense  Advance  Research 
Projects  Agency  to  demonstrate  the 
utility  of  DIS  for  combat  identification. 
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CHAPTER  10 
WORKING  GROUP  9 

SIMULATION  PROTOTYPING  TO  SUPPORT  ACQUISITION 

Richard  E.  Garvey,  Jr  FS 


The  focus  of  Working  Group  9  was  to  exam¬ 
ine  prototyping  using  Distributed  Interactive 
Simulation  (DIS)  environments  to  support  the 
materiel  acquisition  process. 

On  29  September  1992,  we  met  and  spent 
about  an  hour  in  a  "get  acquainted"  mode. 
Each  person  explained  his  current  work  and 
reasons  for  coming  to  the  DIS  Mini-Sympo¬ 
sium.  We  then  discussed  the  purpose  of  our 
working  group,  the  required  format  of  our 
report,  and  the  agenda  for  the  following  day. 
We  ended  by  organizing  into  sub-groups  on 
the  following  topics:  Uses  of  DIS,  Limita¬ 
tions  of  DIS,  Required  DIS  Enhancements, 
and  Technical  Considerations. 

On  30  September  1992,  we  began  with  the 
presentation  of  two  papers: 

•  "Simulation  Prototype:  the  LOS  AT 
Weapon  System"  was  presented  by 
Mr.  Gregory  B.  Tackett,  U.S.  Army 
Missile  Command. 

•  "DIS  Prototyping  for  Intelligent  Sub¬ 
systems"  was  presented  by 
Dr.  Alton  L.  Gilbert,  Technical  Solu¬ 
tions,  Inc. 

Following  these  presentations,  we  had  a 
general  discussion  with  the  entire  working 
group  participating.  We  then  divided  into 
sub-group  discussions.  Each  sub-group 
reported  on  its  findings  to  the  entire  working 
group  which  provided  the  opportunity  for 
exchanging  comments  and  clarifying  main 
points.  At  the  conclusions  of  the  presenta¬ 


tions,  the  four  sub-group  chairs  and  the 
working  group  chair  met  to  discuss  and 
assemble  the  report  that  would  be  delivered 
to  the  Mini-Symposium  the  next  morning. 

Summary  findings  of  each  sub-group  along 
with  general  conclusions  and  recommenda¬ 
tions  follow  below. 

FINDINGS 

Sub-Group  A,  which  was  chaired  by  Mr. 
Richard  E.  Helmuth  of  Douglas  Aircraft 
Company,  reached  the  following  conclusions 
regarding  principal  uses  of  DIS  for  simula¬ 
tion  prototyping  (Pre-Milestone  0  through 
Milestone  II): 

•  Development  of  consistent  MOE/MOP 
for  ALL  communities 

•  Requirements  definition/refinement 

•  Research 

-  C2 

-  Support  (CS  and  CSS) 

-  Behavioral  phenomena 

-  Alternate  technologies 

Sub-Group  B,  which  was  chaired  by  James 
P.  Hogarty  of  General  Research  Corporation, 
came  up  with  the  following  Current  Limita¬ 
tions  of  DIS  for  simulation  prototyping: 

•  Configuration  control/documentation 

•  W&A 

•  Testbed  availability 

•  Limited  warfare  "arena"  (NOT  seam¬ 
less) 
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•  Resources 

•  Community  acceptance 

Sub-Group  C,  which  was  chaired  by  Mr. 
Gregory  B.  Tackett  of  the  U.S.  Army 
MICOM,  concluded  that  the  following  are 
the  major  Required  Enhancements  of  DIS  for 
simulation  prototyping: 

•  Environment 

-  Natural 

—  Weather,  Day/night 
--  Fidelity,  Correlation 
—  Standard  data  base  library 
—  Rapid  terrain  DB  generation 

-  Man-Made 

—  Battlefield  obscurants 
—  Dynamic  terrain 
--  EW/Commo 

•  DIS  System 

-  Higher  fidelity  displays 

-  Number  of  entities 

•  Systems 

-  Sensor  algorithms/signatures 

-  Artillery  and  effects 

-  Modular  system  component  libraries 

•  Data 

-  Approved  weapons  performance  data 

-  Approved  scenarios 


Sub-Group  D,  which  was  chaired  by  Dr. 
Ronald  C.  Hofer  of  the  U.S.  Army 
STRICOM  developed  the  following  list  of 


•  Classified  operations/shared  data  bases/ 
communication  services 

•  Aggregation/Disaggregation 

•  Linkages  to  Higher  Order  Models 

•  VV&A 

•  Linkages  between  Combat  Develop¬ 
ment,  Materiel  Development,  Analytic, 
and  Operational  Test  Communities 


CONCLUSIONS 


•  DIS  prototyping  can  be  valuable  to  the 
materiel  acquisition  process 

•  Acceptance  will  depend  on  enhance¬ 
ments,  disciplined  use,  and  education 

•  A  shared  investment  by  DOD  is  re¬ 
quired 

RECOMMENDATIONS 


•  That  DOD  invest  in  the  further  devel¬ 
opment  and  enhancement  of  DIS  tech¬ 
nology 

•  That  DOD  change  the  materiel  acquisi¬ 
tion  process  to  include  DIS  prototyping 
as  appropriate 
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CHAPTER  11 
WORKING  GROUP  10 
COST/BENEFIT/RISK  OF  DIS 
Edward  C.  Brady,  FS 


Working  Group  10  of  the  DIS  Mini-Sympo¬ 
sium  was  charged  with  the  exploration  of  the 
costs,  benefits,  and  risks  of  DIS.  Over  20 
attendees  at  the  mini-symposium  participated 
in  the  working  group’s  discussions  and  delib¬ 
erations,  which  covered  a  very  wide  range  of 
potential  applications  and  issues. 

DEFINE  DIS  AND  ITS  APPLICATIONS 

The  working  group  found  that  it  needed  a 
definition  of  DIS.  The  group  included 
representatives  from  multiple  application 
areas,  and  their  vision  of  DIS  appeared  to  be 
much  broader  than  that  comprised  of  manned 
weapon  system  simulators,  SAFOR,  and 
associated  protocols.  The  group  defined  DIS 
to  be: 

"Any  interactive  combination  of 
virtual,  constructive  and  live 
simulations. " 

This  definition  encompasses  SIMNET  as 
originally  implemented,  but  it  would  also 
include  situations  such  as  CBS  linked  to 
TACSIM  and  ENWGS,  SIMNET  linked  to 
EAGLE,  and  SIMNET  linked  to  units  exer¬ 
cising  at  the  NTC.  Similar  examples  could 
be  drawn  from  the  Navy  and  the  Air  Force. 

In  addressing  the  issues  of  cost,  benefit,  and 
risks  the  group  included  applications  that 
ranged  from  macro  to  micro  levels.  The 
macro  levels  began  with  the  "road  to  war” 
and  included  such  applications  as  crisis 
management,  industrial  readiness,  mobiliza¬ 


tion  conflict,  demobilization,  and  reconstitu¬ 
tion.  Micro  levels  included  system  on  sys¬ 
tem  or  item  level  applications,  ranging  from 
one-on-one  duels  to  small  unit  engagements 
in  a  combined  arms  context. 

The  group  developed  the  categorization 
scheme  illustrated  in  Figure  11-2  —  a  DIS 
architecture  organized  according  to  echelon, 
continuum  of  conflict,  and  application.  It 
was  the  consensus  of  the  group  that  all  ele¬ 
ments  of  this  architecture  are  valid  candidates 
for  DIS.  It  was  further  noted  that  experience 
to  date  is  narrow:  primarily  collective  train¬ 
ing  at  the  small  unit  level  in  hostilities.  The 
benefits  of  the  technology  clearly  can  and 
should  be  extended  throughout  the  architec¬ 
ture. 

Given  the  architecture  described  above  the 
working  group  concluded  that  DIS  offers 
significant  benefits.  The  group  believes, 
however,  that  it  is  a  matter  of  major  concern 
that  the  range  of  potential  applications  and, 
thus,  benefits  is  much  larger  than  the  very 
limited  applications  which  exist  currently. 
Moreover,  rigorous,  quantitative  evaluation 
of  benefits  for  existing  applications  is  not 
widespread;  most  evidence  is  anecdotal. 

The  working  group  is  convinced  that  DIS 
applications  will  be  valuable  and  that  the 
sooner  the  applications  are  available,  the 
better.  Moreover,  the  group  urges  caution. 
Resources  are  scarce.  DIS  overall  will 
compete  for  funding  and  within  DIS  various 
applications  will  make  claims  on  limited 


51 


COST/BENEFIT/RISK 
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FIGURE  11-2.  DIS  Architecture 


resources.  Given  this  situation,  it  is  impor¬ 
tant  to  have  an  overall  strategy  or  plan  — 
what  to  do,  why,  and  when.  That  plan  does 
not,  in  the  group’s  view,  exist.  We  recom¬ 
mend  that  such  a  plan  be  developed. 

COST/BENEFIT/RISK 

In  initiating  its  discussions,  the  working 
group  examined  a  set  of  hypotheses  concern¬ 
ing  costs,  benefits,  and  risks.  Broadly  speak¬ 
ing,  the  hypotheses  were  that: 

•  There  were  no  major  impediments  to 
estimating  the  costs  of  DIS 
applications,  but  that  the  effort  had  not 
yet  been  made; 

•  There  were  significant  benefits  to  DIS, 
but  that  benefits  of  specific  applications 
had  not  yet  been  measured;  and 

•  Risks  associated  with  costs,  schedule, 
and  performance  while  not  exhaust¬ 
ively  identified  appeared  to  be  manage¬ 
able. 


In  its  discussions  and  deliberations  the  work¬ 
ing  group  uncovered  no  reasons  to  reject 
these  hypotheses.  However,  the  following 
risks  were  identified  as  worthy  of  note: 

•  The  expectations  that  DIS  is  immedi¬ 
ately  available  are  based  upon  demon¬ 
strations  and  prototypes  in  limited 
numbers  with  limited  capabilities.  If 
past  experience  holds,  hardware  and 
software  which  are  affordable  and 
fieldable  in  larger  quantities  may  not 
be  available  for  a  while. 

•  The  full  costs  of  DIS  have  not  yet  been 
determined  and  many  costs,  particu¬ 
larly  operating  costs  such  as  configura¬ 
tion  management,  data  management, 
support  staffing,  verification,  and  vali¬ 
dation,  have  not  yet  been  addressed  in 
detail.  These  hidden  costs  may  be 
large  and  may  not  fall  naturally  into 
any  one  organization's  budget. 

•  The  development  of  models  and  simu¬ 
lations  for  DIS  applications  will  pose  a 
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series  of  challenges  since  it  will  be 
necessary  to  break  new  ground.  Vali¬ 
dation  of  these  new  developments  will 
be  very  challenging.  SAFOR  is  a  good 
example  of  this  -  the  costs  of  specific 
applications  and  the  benefits  depend 
upon  the  validity  and  scope  of  associ¬ 
ated  SAFOR.  SAFOR  technology  is 
not  yet  mature.  Similarly,  the  question 
of  matching  fidelity  to  application 
remains  open  and  must  be  resolved. 

•  Among  developers  of  DIS  and  among 
potential  users,  there  are  perceptions 
that  DIS  replaces  tools  currently  used 
in  all  applications.  This  perception  is 
not  correct.  DIS  will  augment  other 
tools  and  approaches.  Care  must  be 
taken  if  we  are  to  avoid  premature 
acceptance  -  or  rejection  -  of  DIS  in 
various  applications. 

CONCLUSION 

In  summary,  the  working  group  concluded 


that  the  benefits  of  DIS  may  be  substantial. 
These  benefits  must  be  defined,  data  must  be 
collected,  and  quantitative  estimates  devel¬ 
oped.  There  is  evidence  that  initial  steps 
toward  this  goal  are  being  taken  (an  interest¬ 
ing  briefing  on  an  ongoing  project  was  pre¬ 
sented).  They  should  be  emphasized  and 
resourced.  With  respect  to  costs  the  working 
group  believes  that  there  is  no  methodologi¬ 
cal  barrier  to  estimating  costs,  but  that  com¬ 
mitment  of  resources  is  required.  Attention 
must  be  focused  on  hidden  costs  which  to 
date  have  largely  been  ignored.  Finally,  the 
group  concluded  that  risks  are  real  but  man¬ 
ageable.  Further,  the  group  noted  the  poten¬ 
tial  existence  and  danger  of  a  mismatch 
between  perceptions  and  expectations  and  the 
likely  near  term  and  longer  term  results  of 
DIS  programs.  It  is  important  to  proceed 
with  and  to  implement  DIS.  It  is  equally 
important  that  existing  tools  and  methods  not 
be  rejected  prematurely  or  inappropriately 
because  of  misperceptions  regarding  the 
benefits  and  costs  of  DIS  applications. 
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Appendix  A 

GLOSSARY  OF  ACRONYMS  AND  ABBREVIATIONS 


ADS 

Advanced  Distributed  Simulation 

ALSP 

Aggregate  Level  Simulation  Protocol 

ASA(RDA) 

Assistant  Secretary  of  the  Army  (Research,  Development  and  Acquisition) 

C2 

Command  and  Control 

CAU 

Cell  adaptor  unit 

CCTT 

Close  Combat  Tactical  Trainer 

CD 

Combat  Development 

CSSTSS 

Combat  Service  Support  Training  Simulation  System 

COTS 

Commercial  off  the  shelf 

CGF 

Computer  Generated  Forces 

CBS 

Corps  Battle  Simulation 

COEA 

Cost  and  Operational  Effectiveness  Analysis 

DARPA 

Defense  Advanced  Research  Projects  Agency 

DSI 

Defense  Simulation  Internet 

DOF 

Degree  of  freedom 

DoD 

Department  of  Defense 

DIS 

Distributed  Interactive  Simulation 

EOA 

Early  Operational  Assessment 

EW 

Electronic  warfare 

FFRDC 

Federally  funded  research  and  development  center 

FOT&E 

Follow-on  OT&E 

FYTP 

Five  year  test  plan 

GOTS 

Government  off  the  shelf 

IOT&E 

Initial  OT&E 

LANL 

Los  Alamos  National  Laboratory 

MAIS 

Major  Automated  Information  Systems 

MOR 

Military  Operations  Research 

MORS 

Military  Operations  Research  Society 

O&M 

Operations  and  maintenance 

OASD(PA&E) 

Office  of  Assistant  Secretary  of  Defense  (Program  Analysis  &  Evaluation) 

OSD 

Office  of  the  Secretary  of  Defense 

OT&E 

Operational  test  and  evaluation 

OR 

Operations  Research 

OUE 

Operational  Utility  Evaluation 

PDUs 

Protocol  data  units 

RFPs 

Requests  for  proposals 

SAFOR 

Semi-automated  forces 

SIMNET 

Simulation  network 

STRICOM 

U.S.  Army  Simulation,  Training  and  Instrumentation  Command 

T&E 

Test  and  evaluation 
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TRAC 
TRAC  OAC 
TRADOC 
USAAC  (DCD) 
W&A 


U.S.  Army  Training  and  Doctrine  Command  Analysis  Command 

U.S.  Army  TRAC  Operations  Analysis  Center 

U.S.  Army  Training  and  Doctrine  Command 

US  Army  Armor  Center  Directorate  of  Combat  Development 

Verification,  validation,  and  accreditation 
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APPENDIX  B 
TERMS  OF  REFERENCE 

MORS  Mini-Symposium: 

Distributed  Interactive  Simulation  (DIS) 

8  July  1992 

1 .  Goal:  The  first  goal  of  this  mini-symposium  is  to  familiarize  the  military  operations  research 
community  with  the  concept  and  capabilities  of  a  Distributed  Interactive  Simulation  (DIS)  and 
synthetic  combat  environments.  The  second  goal  is  to  have  the  participants  explore  the  use  of  these 
capabilities  to  meet  their  operations  research  and  analysis  needs. 

2.  Background: 

a.  Synthetic  Combat  is  a  computer  simulation  that  allows  groups  of  users  immersion  in  a 
simulated  battle.  Synthetic  combat  environments  allow  a  computer-human  interface  where  analysts 
may  navigate  in  information-based  space  to  explore  systems  and  combat  operations  without  going 
to  the  field. 

b.  The  DARP  A/Army  SIMulator  NET  work  (SIMNET)  is  the  current  proof-of-principle. 
SIMNET  has  proven  itself  as: 

(1)  a  useful  man-in-the-loop  combat  simulation; 

(2)  a  tool  for  exploring  operational  concepts,  unit  tactics  and  procedures; 

(3)  and  a  means  of  introducing  very  early  manned  simulators  of  developmental  weapon 
systems  into  simulated  combat  environment. 

The  core  notion  of  SIMNET,  a  DIS  system,  has  been  designated  as  a  key  element  of  the  DDR&E’s 
Science  and  Technology  area  number  six  (Synthetic  Environments).  This  thrust  includes  technology 
for  acquisition,  combat  developments,  and  training  and  readiness. 

c.  Future  DIS  systems  must  interoperate  with  different  SIMULATORS  and  SIMULATIONS 
at  different  levels  of  resolution  and  aggregation.  Real-time  dynamic  objects  must  allow  modification 
of  terrain  and  cultural  features  distributed  in  the  DIS  network.  Protocol  data  units  may  allow  a  wide 
variety  of  operational  systems,  manned  simulators  and  closed-form  simulations  to  be  interoperable 
in  real-time.  DIS  protocols  require  a  style  of  interaction  that  is  different  from  traditional  modeling 
and  simulation.  The  simulation  and  modeling  architectures  are  radically  different.  The  Military 
Operations  Research  Society  (MORS)  Mini-Symposium  is  an  excellent  organizational  vehicle  to 
stimulate  thought  and  work  in  this  area. 
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3.  Objectives: 


a.  The  purpose  of  this  MORS  mini-symposium  is  to  explore  military  operations  research 
applications  of  DIS  technology  and  its  environments.  The  symposium  will  provide  a  forum  to 
examine  DIS  technology  applications  to  analysis,  testing,  and  training.  The  major  objectives  are  to 
provide: 


(1)  a  learning  experience; 

(2)  examine  the  utility  and  limitations  of  the  DIS  simulation  environments;  and 

(3)  develop  issue  areas  in  the  use  of  synthetic  environments  for  bench  marking  and  data 
collection. 

b.  Within  these  general  objectives,  the  forum  will  specifically  address  the  following  subjects: 

(1)  explore  the  role  of  DIS  in  force  planning  and  contingency  analysis; 

(2)  examine  the  use  of  DIS  to  analyze  adaptive  tactics,  techniques  and  procedures; 

(3)  examine  the  role  of  DIS  in  defense  acquisition  systems  and  early  life  cycle  prototyping 
and  testing  of  new  concepts.  Use  of  selective  fidelity  and  rapid  prototyping,  and  computer 
networking  for  concurrent  engineering  will  also  be  examined; 

(4)  analyze  the  use  of  DIS  for  historical  representation  and  analysis  of  battles;  and 

(5)  identify  useful  extensions  for  DIS. 

4.  Approach:  The  mini-symposium  will  achieve  the  above  objectives  through  a  multi-layered 
approach  which  will  include: 

a.  A  read-ahead  package  will  be  sent  to  all  participants  to  provide  background  information 
and  an  overview  of  DIS  capabilities  and  objectives. 

b.  Plenary  sessions  featuring  keynote  speakers  who  will  address  visions  for  DIS  synthetic 
combat  as  well  as  assessments  of  DIS  limitations  and  current  applications  and  lessons  learned. 
Working  group  introductions  and  a  social  mixer  will  close  out  the  first  day  of  the  mini-symposium. 

c.  Working  groups'.  Working  group  tracks  will  meet  on  the  second  day  of  the  mini¬ 
symposium.  The  working  groups  are: 

(1)  Military  Analysis  (investigating  battle  outcomes,  tradeoff  studies), 

(2)  Test  and  Evaluation, 

(3)  Operations  Planning  and  Rehearsal, 

(4)  Logistics,  Mobilization,  and  Sustainment, 

(5)  Force  Developments  (force  structure,  mission  area  analyses), 

(6)  Combat  Developments  (doctrine,  tactics,  procedures), 

(7)  Training  and  Readiness 

(8)  Requirements  Development  and  Definition, 

(9)  Simulation  Prototyping  to  Support  Acquisition,  and 
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(10)  Cost/Benefit/Risk  of  DIS. 


Each  working  group  will  identify  uses,  limitations  and  enhancements  of  DIS  to  support  their  specific 
needs.  Each  group  is  encouraged  to  address  human  performance  and  behavioral  concerns, 
environmental  requirements  such  as  terrain  resolution,  and  data  collection  and  reduction  issues  in 
support  of  performance  and  effectiveness  measures. 

d.  Demonstrations'.  During  the  second  day  of  the  mini-symposium,  a  War  Breaker 
demonstration  on  the  acquisition  and  attack  of  critical  mobile  targets  will  be  given  at  the  Simulation 
Center,  Institute  for  Defense  Analyses,  1801  N.  Beauregard  Street,  Alexandria,  VA. 

e.  On  the  morning  of  the  third  day,  a  wrap-up  session  will  be  held  for  the  working  groups  to 
present  their  findings  and  to  identify  any  possible  follow-on  efforts  appropriate  for  MORS  support. 

f.  An  optional  tutorial  for  DIS  standards  will  be  offered  the  evening  prior  to  the  start  of  the 
mini-symposium. 

5.  MembershiprThe  representation  is  expected  to  be  50%  DoD  and  50%  industry  and  academia. 
Congressional  staffers  and  General  Accounting  Office  (GAO)  personnel  will  be  invited.  Working 
groups  should  be  limited  to  25  people.  Working  group  chairpersons  should  be  considered  subject 
matter  experts  in  their  session  area.  Membership  in  the  working  groups  will  be  controlled  by  the 
working  group  chairpersons. 

6.  Product:  A  briefing  will  be  prepared  for  the  sponsors  of  the  meeting  to  report  findings, 
conclusions,  and  recommendations.  A  proceedings  will  be  prepared  containing  an  executive 
summary,  each  working  group’s  report,  and  copies  of  papers  and  briefings  presented. 

7.  Proponents:The  Army  and  Navy  have  agreed  to  be  co-proponents  for  the  mini-symposium. 
The  Air  Force,  OSD,  and  the  Joint  Staff  have  expressed  supportive  interest. 

8.  Planning  and  Organizational  Committee: 

General  Chair:  Dr.  Henry  C.  Dubin 

US  Army  OPTEC 
Park  Center  IV,  4501  Ford  Ave. 

Alexandria  VA  22302-1458 
703-756-2365  FAX  703-756-0779 

Deputy  Chairs:  Mrs.  Margie-Elaine  Wolchak/Mr.  Greg  G.  Guernsey 

US  Army  OPTEC 
Park  Center  IV,  4501  Ford  Ave. 

Alexandria  VA  22302-1458 
703-756-1818  FAX  703-756-7586 
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Members: 


Mr.  Ed  Brady 
Strategic  Perspectives 
7704  Lakeloft  Court 
Fairfax  Station  VA  22039 
703-250-6338  FAX  703-250-3637 

CAPT  Tom  Flaherty,  USN 
CNO  OP-73 
The  Pentagon 
Washington  DC  20350 
703-614-1321  FAX  703-695-5698 

Mr.  Richard  Garvey 
Greenbrier  Group 
310  Forest  Trace 
RadcliffKY  40160 
502-877-5738  FAX  502-877-0325 

Dr.  Ron  Hofer 

STRICOM 

12350  Research  Pkwy 

Orlando  FL  32826-3276 

407-380-8077  FAX  407-380-4258 

Mr.  Kenneth  Lavoie 
Air  Force  Wargaming  Center 
AUCADRE/WGT  BLDG  1406 
Maxwell  AFB  AL  361 12 
205-953-6528  FAX  205-953-2593 

Mr.  John  Mills 

Navy  DIS  Agent 

Naval  Training  Systems  Center 

Code  PDB8 

12350  Research  Plcwy 

Orlando  FL  32826 

407-380-8367  FAX  407-380-4412 

Dr.  Jim  Metzger 
OSD  PA&E 
Land  Forces  Division 
Pentagon,  Room  2B256 
Washington  DC  20301 
703-697-3521  FAX  73693-5707 
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Col  James  Shiflett 

Defense  Modeling  and  Simulation  Office 
1901  N.  Beauregard  Street,  #504 
Alexandria  VA  22311 
703-998-0660  FAX  703-998-0667 

Mr.  Clayton  Thomas,  FS 
HQ  USAF  AFSAA 
Pentagon,  Room  1E386 
Washington  DC  20330 
703-697-4300  FAX  703-697-3441 

Dr.  Ben  Wise 

Bolt,  Beranek,  and  Newman 
33  Moulton  Street 
Cambridge  MA  02138 
617-873-3511  FAX  617-873-4315 


Administrative 

Coordinators:  Mr.  Richard  Wiles/Ms.  Natalie  Addison 
Military  Operations  Research  Society 
Landmark  Towers,  Suite  202 
101  S.  Whiting  Street 
Alexandria  VA  22304 
703-751-7290  FAX  703-751-8171 
E-mail:  rwiles@dgis.dtic.dla.mil 

9.  Administration: 


Title:  Distributed  Interactive  Simulation  (DIS)  a  Synthetic  for  Military  Operations 

Research 

Proponent:  US  Army  and  US  Navy.  The  Executive  Agent  will  be  the  US  Army  Operational 
Test  and  Evaluation  Command.  MORS  will  provide  administrative  and  logistics 
support. 

Dates:  29  Sept  - 1  Oct  92 


Fee:  $150.00  (Federal  Government) 

$300.00  (Others) 


Attendance:  Limited  to  250  attendees 


Classification:  Unclassified 
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APPENDIX  C 
PARTICIPANTS1 


Anita  R  Adams 

The  MITRE  Corporation 

2166  KINGS  GARDEN  WAY 

Falls  Church  VA  22043 

(703)-883-1389 

FAX:  (703)-883-3343 

EMAIL:  ANITA@MITRE.ORG 

James  L  Adams 

Richard  S.  Carson  &  Assoc,  Inc 
4350  East- West  Highway,  Suite  1000 
Bethesda  MD  20814 
(301)-656-4565 
FAX:  (301)-656-4806 

COL  Lonnie  B  Adams  III 
Headquarters  Defense  Nuclear  Agency 
6801  Telegraph  Road 
Alexandria  VA  22310-3398 
(703)-325-7707  DSN:  221-7077 
FAX:  (703)-325-2951 

Natalie  S  Addison 

Military  Operations  Research  Society 
101  S  Whiting  Street 
Suite  202 

Alexandria  VA  22304-3483 
(703)-75 1-7290 
FAX:  (703)-75 1-8171 


DR  Irving  N  Alderman 
US  Army  Research  Institute 
Attn:  PERI-II 
5001  Eisenhower  Ave 
Alexandria  VA  22333-1535 
(703)-274-8776  DSN:  284-8776 
FAX:  (703)-274-3268 

EMAIL:  alderman@alexandria-emh2.army.mil 

John  R  Armendariz 
USAADASCH 
ATTN:  ATSA-CDF 
Fort  Bliss  TX  79916-3802 
(915)-568-1238  DSN:  978-1238 
FAX:  (915)-568-2647 

James  H  Atkinson  Sr 
ILLGEN  Simulation  Technologies,  INC 
250  Storke  Road 
Suite  10 

Goleta  CA  93117 
(301)-805-4439 
FAX:  (301)-805-5864 

William  J  Babilon 
Decision  Science  Applications  Inc 
1 1 10  N  Glebe  Road 
Suite  400 

Arlington  VA  22201 
(703)-243-2500 
FAX:  (703)-875-9882 


George  W  Aitken 
OptiMetrics,  Inc 
5402  Baychester  Court 
Alexandria  VA  22310 
(703)-922-6846 
FAX:  (703)-922-5613 


1  Addresses  given  are  current  in  the  MORS  database  as  of  July  31,  1995. 
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Robert  F.  Bachert 

Advanced  Systems  Analysis  Directorate 
DCS,  Development  Planning 
ASC/XRE 

Wright-Patterson  AFB  OH  45433-6573 
(513)-255-5289  DSN:  785-5289 
FAX:  (513)-476-7603 
EMAIL:  bachertrf@p4.ams.wpafb.af.mil 

Warren  E  Bailey 

Naval  Air  Warfare  Center  Aircraft  Div 

NAWCAD 

FTEG,  Code  RE80 

Patuxent  River  NAS  MD  20670 

(301)-863-1944  DSN:  326-1944 

COL  James  T  Baird 
OSD(PA&E),  ODASD(TA&P) 

Europe  &  Pacific  Forces  Div 
Pentagon,  Room  2C270 
Washington  DC  20301-1800 
(703)-697-6415 
FAX:  (703)-693-5707 

Edward  F  Baker  Jr 
Naval  Surface  Warfare  Center 
10901  New  Hampshire  Ave 
Silver  Spring  MD  20903-5000 
(301)-394-1269  DSN:  290-1269 

James  R  Bambery 
Vector  Research,  Inc 
POBox  1506 
Ann  Arbor  M3  48106 
(313)-973-9210 
FAX:  (313)-973-7845 

Melanie  W  Barry 

General  Research  Corporation 

1 900  Gallows  Road 

Vienna  VA  22182-3865 

(703)-243-4822 

FAX:  (703 )-243 -4821 


MAJ  Michael  J  Barton 
(410)-278-4918 
FAX:  (410)-278-6242 
EMAIL:  mbarton  @  brl.mil 

Michael  F.  Bauman 

US  Army  TRADOC  Analysis  Center 

ATTN:  ATRC 

255  Sedgwick  Ave 

Fort  Leavenworth  KS  66027-2345 

(913)-684-5 132  DSN:  552-4689 

FAX:  (913)-684-6894 

EMAIL:  baumanm@tracer.army.mil 

James  R  Behne 

TRADOC  Analysis  Command-Lee 
Attn:  ATRC-LS 
Fort  Lee  VA  23801-6140 
(804)-765-1830  DSN:  539-1838 
FAX:  (804)-539-1456 

DR  Herbert  H  Bell 

William  M.  Benson 

US  Army  Operational  Test  &  Eval  Comd 

ATTN:  CSTE-TSM 

4501  Ford  Ave 

Alexandria  VA  22302-1458 

(202)-75 6-2487 

Albert  C  Berrie 
FAAC  Inc 
825  Victors  Way 
Ann  Arbor  MI  48 1 08 
(3 13)-76 1-5836 
FAX:  (313)-761-5368 

Amy  Bettencourt 
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James  N  Bexfield  FS 
Institute  for  Defense  Analyses 
1801  N.  Beauregard 
Alexandria  VA  223 11-1772 
(703)-845-2107  DSN:  289-1825 
FAX:  (703)-578-2813 
EMAIL:  jbexfiel@ida.org 

Barbara  J  Birks 

McDonnell  Douglas-Aircraft  Co 

B33/L4/MC  0341550 

PO  Box  516 

St.  Louis  MO  63166 

(314)-233-3120 

FAX:  (314)-777-2491 

EMAIL: 

a23  7250%etd.decnet@mdcgwy.mdc.  com 

DR  Barbara  A  Black 

US  Army  Research  Institute 

Fort  Knox  Field  Unit 

Attn:  PERI-IK 

Fort  Knox  KY  40121-5620 

(502)-624-3450  DSN:  464-3450 

FAX:  (502)-624-81 13 

EMAIL:  black@alexandria-emh2.army.mil 

Robert  J  Blizek 

Benjamin  D  Blood 
Coleman  Research  Corporation 
3045  Technology  Pkwy 
Orlando  FL  32826 
(407)-249-7717 
FAX:  (407)-3  81-3980 

Irvin  E  Boyles 

US  Army  Operational  Test  &  Eval  Comd 
4501  Ford  Avenue 
Attn:  OEC-ECC-A 
Alexandria  VA  22302-1458 
(703)-756-2166  DSN:  289-2166 
FAX:  (703)-756-2787 


John  Brabbs 

US  Army  TARDEC 

AMSTA-RVI 

Warren  MI  48397-5000 

(3 13)-574-6242  DSN:  786-6242 

FAX:  (313)-574-5008 

LTC  William  L  Braddy  Jr 
The  Joint  Staff 
Integration  and  Assessment 
Pentagon,  Room  1D964 
Washington  DC  20318-8000 
(703)-697-0557  DSN:  557-0557 
FAX:  (703)-6 14-6601 

Edward  C  Brady  FS 
Strategic  Perspectives,  Inc. 

7704  Lakeloft  Court 
Fairfax  Station  VA  22039 
(703)-250-6338 
FAX:  (703)-764-2699 
EMAIL:  brady@cerfiiet.com 

CPT  William  C.  Branley 

US  Army  Artificial  Intelligence  Center 

The  Pentagon,  Room  1D659 

Washington  DC  203 10 

(703)-697-7250 

FAX:  (703)-6 14-6908 

EMAIL:  branley@pentagon-ai.army.mil 

Gilbert  MF  Brauch  Jr 

Consultant 

LMI 

6400  Goldsboro  Road 
BethesdaMD  20817-5887 
(703)-917-7383 
FAX:  (3  01  >-320-4701 

GerlofBril 

Physics  &  Electronics  Lab  TNO 
PO  Box  96864,  2509  JG  the  Hague 
The  Netherlands 
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William  A  Brinkley 

Teledyne  Brown  Engineering 

PO  Box  070007 

300  Sparkman  Dr,  MS  170 

Huntsville  AL  35807-7007 

(205)-726-5857 

FAX:  (205)-726-2241 

EMAIL:  tony.brinkley@pobox.tbe.com 

Peter  S  Brooks 

Institute  for  Defense  Analyses 

1801  N.  Beauregard  Street 

Alexandria  VA  223 1 1 

(703)-845-2170 

FAX:  (703)-845-6809 

EMAIL:  pbrooks@ida.org 

Wilbert  I  Brooks  Ir. 

US  Army  Materiel  Systems  Analysis  Acty 
Attn:  AMXSY-GA 

Aberdeen  Proving  Ground  MD  21005-5071 
(410)-278-4946  DSN:  298-4608 
FAX:  (301)-278-3294 
EMAIL:  wbrooks@amsaa.brl.mil 

Robert  B  Buchanan 
SAIC 

1710  Goodridge  Road 
MS  2-8-1 

McLean  VA  22102 

(703)-821-4576 

FAX:  (703)-893-2439 

EMAIL:  Bob_Buchanan@cpms.saic.com 

Charles  D  Burdick 
Loral  Advanced  Project? 

PO  BOX  1339 
Manassas  VA  22110-1339 
(703)-759-1407 
FAX:  (703)-759-1450 


Peter  C  Burggren 
Cubic  Defense  Systems,  Inc 
MS  2-4 

9333  Balboa  Ave 
San  Diego  CA  92123 
(619)-277-6780 
FAX:  (619)-277-9524 

William  R  Burris 

TRW  Omnibus  Program  Office 

Crystal  Square  5,  Suite  202 

1755  Jefferson  Davis  Hwy 

Arlington  VA  22202 

(703)-920-8995 

FAX:  (703)-920-9440 

EMAIL:  wburris@tecnetl.jcte.jcs.mil 

Walter  G  Butler 

TRAC-WSMR 

Attn:  ATRC-WGA 

White  Sands  Missile  Range  NM  88002 

(505)-678-4212  DSN:  258-4212 

FAX:  (505)-678-5104 

EMAIL:  butlerw@wsmr-emh91.army.mil 

DR  Monti  D  Callero 

RAND 

1700  Main  St 

Santa  Monica  CA  90407-2138 
(3 1 0)-3  93  -04 1 1 
FAX:  (3 1 0)-3  93  -48 1 8 
EMAIL:  monti@rand.org 

Fred  P  Campo 
Northrop  Aircraft  Corp 
One  Northrop  Avenue 
Hawthorne  CA  90250 
(310)-332-8705 
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Keith  R  Carson 

HQ  TRADOC 

ODCS  Analysis 

Attn:  ATAN-ZA 

Fort  Monroe  VA  23651-5000 

(804)-728-5803  DSN:  680-5803 

FAX:  (804)-727-4394 

EMAIL:  carsonk@monroe-emhl.army.mil 

Thomas  W  Cash 
Nichols  Research  Corportion 
4040  South  Memorial  Pkwy 
Huntsville  AL  35802 
(205)-883-l  170 
FAX:  (205)-880-0367 

William  C.  Cave 
Prediction  Systems,  Inc 
309  Morris  Ave  #G 
Spring  Lake  NJ  07762-1338 
(908)-223-5000 

Wallace  W  Chandler 
US  Army  CAA 
ATTN:  CSCA-RSD 
8120  Woodmont  Ave 
Bethesda  MD  20814-2797 
(301)-295-1643  DSN:  295-1643 
FAX:  (301)-295-1834 

LTC  George  T  Cherolis 
(505)-678-2264  DSN:  258-4300 
FAX:  (505)-678-5104 
EMAIL:  cherolig@wsmr-emhl.army.mil 

DR  W  Peter  Cherry 

Vector  Research  Inc 

PO  Box  1506 

Ann  Arbor  MI  48106 

(313)-973-9210 

FAX:  (313)-973-7845 

EMAIL:  cherryw@vrinet.com 


Jan  Chervenak 

Dismounted  Warfighting  Center 
Battle  Lab 

ATTN:  ATSH-IWCS 
Fort  Benning  GA  3 1905-5420 
(706)-545-7000  DSN:  835-7000 
FAX:  (706)-545-2517 

Willard  M  Christenson 
Institute  for  Defense  Analyses 
SFRD  Room  517 
1801  N.  Beauregard  St 
Alexandria  VA  223 1 1 
(703)-845-2439 
FAX:  (703)-845-2255 
EMAIL:  wchriste@ida.org 

Julie  C  Chu 

US  Army  Armament  Rsch  Dev  &  Engineer 
Ctr 

ATTN:  SMCAR-ASH 
Picatinny  Arsenal  NJ  07806-5000 
(20 1)-724-7088  DSN:  880-7088 
FAX:  (201)-724-4111 
EMAIL:  jchu@pica.army.mil 

COL  Robert  B  Clarke 
C  S  A/P  AED/ASP  AD 
Pentagon,  Room  3C747 
Washington  DC  203 10-0200 
(703)-697-6047  DSN:  227-6047 
FAX:  (703)-693-6993 

Gerald  C  Cleaver 

Robert  Clover 

Institute  for  Defense  Analyses 
Simulation  Center 
1801  N.  Beauregard  St 
Alexandria  VA  223 1 1-1772 
(703)-845-6807 
FAX:  (703)-845-6809 
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Mark  L  Cole 
SYSCON  Corporation 
Rt  206,  PO  Box  1480 
Dahlgren  VA  22448 
(703)-663-4244 

DR  Darrell  W  Collier 

US  Army  Strategic  Defense  Command 

Crystal  Mall  4,  Suite  900 

1941  Jefferson  Davis  Hwy 

Arlington  VA  22215 

(703)-607-1876 

Wesley  S  Corber 
BDM  International,  Inc 
1501  BDM  Way 
McLean  VA  22102 
(703)-848-6537 
FAX:  (703)-848-6666 

Rene  V  Cormier 

Air  Force  Phillips  Lab 

Director  Office  of  Envir  Simulation 

GP-M 

Hanscom  AFB  MA  01731 
(617)-377-3606  DSN:  478-3606 
FAX:  (617)-377-5974 

Salvatore  J.  Culosi 
Logistics  Management  Institute 
6400  Goldsboro  Road 
BethesdaMD  20817-5886 
(703 )-9 17-7368 
FAX:  (703)-9 17-7471 
EMAIL:  SCULOSI@LMI.ORG 

David  W  Curkendall 

Jet  Propulsion  Laboratory 

MS  138-310 

4800  Oak  Grove  Drive 

Pasadena  CA  91 109-8099 

(8 18)-3  54-2406 

FAX:  (8 1 8)-393-6 141 

EMAIL:  dwc@leslie.jpl.nasa.gov 


J.  Carlton  Daniel 
Topograhic  Engineering  Center 
Telegraph  &  Leaf  Road 
Fort  Bel  voir  VA  22060-5546 
(703)-355-3838 
FAX:  (703 >355-3171 

DR  Paul  K  Davis 
RAND 

Defense  &  Technology  Planning  DTP 
PO  Box  2138 

Santa  Monica  CA  90407-2138 
(3 1 0)-3  93  -04 1 1 
FAX:  (3 1 0)-3 93 -48 1 8 
EMAIL:  Paul  Davis@rand.org 

Brian  K  DeGraw 
US  Army  ARDEC 
SMCAR-FSF-A 
Building  91 

Picatinny  Arsenal  NJ  07806-5000 
(201)-724-5730  DSN:  880-5730 
FAX:  (20 1)-724-5768 
EMAIL:  bdegraw@pica.army.mil 

Michael  B  DeMent 
Booz  Allen  &  Hamilton 
8283  Greensboro  Drive 
McLean  V A  22102-3838 
(703)-902-5819 
FAX:  (703)-902-3374 

Hugh  A  Dempsey 

CSA  Louisiana  Maneuvers  Task  Force 
Box  471 

Fort  Monroe  VA  23  65 1  -5 1 43 
(804)-728-5822  DSN:  680-5822 
FAX:  (804)-728-5826 
EMAIL:  dempseyh@tracer.army.mil 
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CDR  Katherine  S  Derie 
The  Joint  Staff  (J-8) 

CFAD 

Pentagon  Room  1D940 
Washington  DC  203 18-8000 
(703)-695-3156  DSN:  225-3456 

Sylvia  A  Diaz 

HQ  Department  of  the  Army 
OASA(RDA),  ATTN:  SARD-DO 
Pentagon,  Room  3D486 
Washington  DC  203 10-0103 
(703)-6 14-5920  DSN:  224-5920 
FAX:  (703)-693-2385 

DR  Philip  C  Dickinson 
E-Systems,  Inc.  (CAPA) 

10530  Rosehaven  Street,  Suite  200 
Fairfax  VA  22030-2886 
(703)-352-0300 
FAX:  (703)-691-3067 

Walton  C  Dickson 

USA  Engineer  Waterways  Exper  Station 
3909  Halls  Ferry  Raod 
Vicksburg  MS  39180-6199 
(60 1)-63 4-2452 
FAX:  (601)-634-3680 

Robert  Dienes 

Combined  Arms  Support  Command 
ATCL-MR 

Fort  Lee  VA  23801-6000 
(804)-734-1614 

Tim  Doane 

General  Physics  Services  Corp 
1919  S.  Eads  Street 
Arlington  VA  22202 
(703)-271-7700 


John  R  Dodd 
Martin  Marietta 
TCCSF 

Kirtland  AFB  NM  87117 
(505)-846-0252 

LTC  Bruce  J  Donlin 
OASA(RDA) 

Attn:  SARD-SM 
Pentagon,  Room  3B465 
Washington  DC  20310-0103 
(703)-697-8643  DSN:  227-8643 
FAX:  (703)-697-3827 

DR  Stephen  Downes-Martin 
Downes-Martin  Associates 
POB  1058 

Northampton  MA  01061 

(413)-582-0183 

FAX:  (413)-586-6765 

EMAIL:  70673.3172@compuserve.com 

DR  Henry  C  Dubin 

US  Army  Operational  Test  &  Eval  Comd 

Attn:  CSTE-ZT 

4501  Ford  Avenue 

Alexandria  VA  22302-1458 

(703)-68 1-9367  DSN:  289-2367 

FAX:  (703 )-68 1-3779 

EMAIL:  dubin@optec.army.mil 

Alan  D  Dunham 

Booz  Allen  &  Hamilton 

4001  North  Fairfax  Drive  #650 

Arlington  VA  22203 

(703)-528-8080 

FAX:  (703)-902-3374 
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Joyce  J  Dyer 

Army  Reserve  Personnel  Center(ARPERCEN) 

9700  Page  Blvd 

Attn:  DARP-ZAP-TR 

Saint  Louis  MO  63132-5200 

(314)-538-2397  DSN:  892-2397 

FAX:  (314)-892-2553 

DR  Gary  K  Eiserman 
Decision  Science  Applications  Inc 
1 1 10  N  Glebe  Road 
Suite  400 

Arlington  VA  22201 
(703)-243-2500 
FAX:  (703)-875-9382 

Anita  S  Eno 

Northrop  Aircraft  Corporation 
One  Northrop  Avenue 
Hawthorne  CA  90250 
(3 10)-33 1-1988 
FAX:  (310)-332-5962 

Richard  A  Esslinger 

NAVSEA 

SEA  06K2 

National  Center  #2,  Room  10W32 
Washington  DC  20362 
(202)-692-1927  DSN:  332-1927 
FAX:  (703)-765-1295 
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Suite  303 

12443  Research  Pkwy 
Orlando  FL  32826 
(407)-382-4580 
FAX:  (407)-3 82-4592 


DR  Herbert  K  Fallin  Jr 
OASA(RDA) 
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103  Army  Pentagon 
Washington  DC  20310-0103 
(703)-697-2653  DSN:  227-2653 
FAX:  (703)-695-9069 

EMAIL:  calahab@pentagon.hqdadss.army.mil 

COL  Bernard  Ferguson 
OSD(DT&E) 

Pentagon,  Room  3E1060 
Washington  DC  20301-3110 
(703)-695-4608 
FAX:  (703)-693-7030 

LTC  Joseph  C  Fernandez 
US  Army  OPTEC 
4501  Ford  Avenue 
Attn:  CSTE-ECS 
Alexandria  VA  22302 
(703)-756-2296 
FAX:  (703)-756-0498 
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General  Electric 

Corporate  R&D  Center,  Bldg  KW,  Room 

C240 

PO  Box  8 

Schenectady  NY  12301 
(518)-387-6570 
FAX:  (5 18)-3 87-5449 

DR  Robert  Finkelstein 

Robotic  Technology,  Inc 

10001  Crestleigh  Lane 

Potomac  MD  20854 

(301)-762-1622 

FAX:  (301)-762-071 6 

EMAIL:  72124.1413@compuserve.com 
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FAX:  (913)-684-9151 
EMAIL:  foxj@tracer.army.mil 
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